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Introduction 

Altmetrics are a hot topic in libraries nowadays, but what do they mean in a practical 
sense for the average librarian’s work? 

This is a topic that I’ve been wrestling with since 2009, when I was introduced to 
altmetrics by way of the PLOS Article-Level Metrics program, which was launching 
just as I joined the company. Just one year out of library school and new to the world 
of scientific publishing, I was entirely taken with the idea of article-level metrics. Of 
course we should understand an article on its own merits, rather than the impact 
factor of the journal it’s been published in! Of course web-based metrics could be 
used to understand how an article has performed on the web, and to supplement our 
understanding of impact provided by citations! 

Fast forward to a year later, when Jason Priem— coiner of the term ‘altmetrics’ —came 
to speak at PLOS about his then-passion: the idea of the decoupled journal. He argued 
eloquently for a future in which publishers like PLOS would be mostly obsolete, where 
third-party services like Pubpeer would handle peer review and other core functions 
of publishing, and where filtering and recommendations would happen by way of the 
social web, rather than editors, thanks to altmetrics. 

“Why rely on people I don’t know to select what I should be reading, when I’ve got a 
Twitter network full of brilliant peers who are a lot better at finding and sharing stuff 
I’m interested in?” Jason asked. 

After Jason’s talk, I began to ponder how all of these “revolutions in scholarly 
communication” applied to libraries. What did we have to offer in the way of Open 
Access publishing or support for newer research services like data archiving, peer 
review, and other functions of scholarly communication? 

More importantly, how would altmetrics apply to the work that librarians do on a day- 
to-day basis, like teaching information literacy concepts, selecting journals and books 
to add to our collections, managing institutional repositories, and so on? 
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With the release of the Altmetrics Manifesto in 2010, I found that my list of questions 
about altmetrics’ applicability to libraries was growing. Soon, plenty of speculative 
articles were being published in the library literature about how libraries could 
potentially use altmetrics, but few concrete answers were being offered about the 
reality of their use in academic libraries. 

Finding answers to librarians’ real-life uses of altmetrics has since driven my career, 
and introduced me to like-minded librarians like Rachel Borchardt, Robin Chin- 
Roemer, Andy Tattersall, Kristi Holmes, Karen Gutzman, and David Scherer (among 
many others), who each share my passion for the promise that altmetrics hold for 
“revolutionizing” library services. They have also formed the basis for a national survey 
I’ve recently completed with Sarah Sutton and Rachel Miles, where we’re going to 
the source— librarians themselves— to understand how librarians use these metrics. I 
share some of the results from that study for the first time in this ebook. 

This ebook is intended to be a starting point for sharing what myself and colleagues 
(fellow librarians Natalia Madjarevic and Amy Rees who are now working at Altmetric) 
have learned about libraries and altmetrics in the six short years since the Altmetrics 
Manifesto was published. We intend for this volume to complement the many excellent 
existing library-oriented resources on altmetrics (chief among them being the books 
Meaningful Metrics: A 21st Century Librarian’s Guide to Bibliometrics, Altmetrics, and 
Research Impact (ACRL Press, 2015) and Altmetrics: A practical guide for librarians, 
researchers and academics (Facet Publishing, 2016)), which are fantastic primers to 
what altmetrics are, the current tools on the market, and how altmetrics relate to 
citation-based metrics. Our book is different in that it provides the “nuts and bolts” 
needed to use altmetrics in a variety of library-land scenarios, including: 

• Making collection development decisions 

• Managing institutional repositories 

• Helping faculty assemble evidence for their tenure & promotion packets 

• Teaching workshops on altmetrics 

We will also share our favorite resources for staying up-to-date on altmetrics research 
and news. 

We have organized this book into three main sections: the General Overview, which 
will share resources related to doing outreach on altmetrics on your campus and how 
to track influence and attention using altmetrics data; Special Issues in Librarianship & 
Altmetrics, which will discuss altmetrics’ applicability to various roles commonly found 
in academic libraries (collection development, scholarly communication librarianship, 
etc); and The Future of Altmetrics, which will share our thoughts on where the field 
of altmetrics is headed, and how you can stay abreast of changes moving forward. 

Some of the scenarios shared in this book use Altmetric’s tools and resources as a 
jumping-off point for illustrating concepts, but we do not intend for you, the reader, 
to stop your exploration of altmetrics there. Other sources for altmetrics data include 
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Scopus, Impactstory, and PlumX, as well as data that can be manually gathered from 
across the web and that’s not included in the existing altmetrics tools on the market 
(data like ResearchGate metrics, Academia.edu metrics, web statistics sourced from 
Google Analytics, and so on). Each tool or data source has its own advantages and 
limitations, and we encourage you to find the tools that best suit your needs. 

We hope that you find this ebook useful to your work and welcome any feedback 
you have. 

Stacy Konkiel 

New Mexico, USA 
Altmetric 
August 2016 
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Outreach on Altmetrics: 

A Complete Guide 

Let’s say that you are your library’s biggest altmetrics advocate. You’re up-to- 
date on the research, have participated in altmetrics-related workshops at library 
conferences, and are confident that altmetrics could be useful to your faculty and 
graduate students, in terms of helping them to understand the value of their research 
and share it with others. 

You organize a brown bag luncheon on altmetrics— hanging posters across campus, 
emailing your colleagues in the library and faculty, preparing an informative 40-minute 
presentation, and anticipating a lively question-and-answer session at the conclusion 
of your talk. 

The day of the luncheon comes and, to your dismay, your only attendees are a curious 
graduate student, a skeptical and argumentative Humanities researcher, and three of 
your colleagues from Reference. 

How did this happen? 

Librarians who are passionate about altmetrics can sometimes forget that the 
concept is still relatively unheard of among faculty and their fellow librarians. Some of 
the important first steps to raising awareness of altmetrics often aren’t taken before 
librarian-advocates ask others to commit resources: attention and time (which few of 
us have much to spare nowadays), or money to purchase altmetrics tools (difficult to 
find without demand from faculty). 

In this chapter, we’ll share some important building blocks for building an 
outreach program on altmetrics at your university’s library: how to start internal 
and external awareness-raising campaigns, assembling reference materials like 
LibGuides, organizing workshops and brown-bags, and, crucially, how to respond 
to misinformation and skepticism. 
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Starting an awareness-raising campaign 

Altmetrics are a relatively new type of data, so it’s no surprise that not a lot of 
researchers nor librarians know about them. Your first order of outreach business is 
to plan small and targeted outreach campaigns to three main stakeholder groups: 
faculty, graduate students, and your fellow librarians. 

Librarian Heather Coates has previously shared her expertise on building successful 
outreach campaigns. She has graciously allowed us to adapt her advice below. 


The following section is adapted from a guest post to the Aitmetric blog contributed 
by Heather Coates, Digital Scholarship & Data Management Librarian at Indiana 
University-Purdue University Indianapolis (IUPUI). 

My perspective as a tenure-track librarian tends to be that of a practitioner- 
researcher. Practically speaking, this means that part of my job is to know how the 
scholarly ecosystem works - to understand how scholarly products are created, 
disseminated, used, curated, and evaluated. 

Over the past three years, I have taught several workshops on using citation 
metrics and altmetrics to demonstrate excellence and impact in promotion and 
tenure (P&T) dossiers. To date, several things have led me to some insights that I 
think are helpful for librarians interested in supporting use of altmetrics: developing 
altmetrics workshops and doing one-on-one consultations; conversations with my 
campus’s Associate Vice Chancellor of Academic Affairs and Director of Faculty 
Enhancement; and the experience of assembling my own dossier. 

Here, I’ll share useful strategies for offering successful altmetrics workshops on your 
own campus, and advice for crafting messaging that resonates with researchers at 
all stages of their careers. 

Get out of the library 

Faculty do not typically think of the library for support in putting together their 
dossiers, so it is crucial to partner with the campus units that faculty do seek out 
for this expertise. 

Luckily for my library, a valuable opportunity fell into our laps. In 2012, librarians 
were invited to work with the Office of Academic Affairs to support faculty in 
gathering evidence for P&T dossiers. This support began with a 2-hour workshop, 
which is now part of a regular series in support of faculty development. Here’s how 
it went. 

We started off in the fall of 2012 with a broad introduction to publication-based 
metrics. It was a fairly traditional library workshop that focused heavily on citation 
metrics from subject and citation databases, plus Google Scholar. However, we 
did describe the various levels of evidence (journal-level, article-level, and author- 
level) and introduce the idea that journal-level metrics are the least relevant to 
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promotion and tenure. We also introduced Impactstory, a researcher profile that 
includes altmetrics data, and we also discussed sources for informal metrics like 
acceptance rates, library ownership counts for books, and indexing status. 

In the first 30 minutes of the workshop, we provided the explanatory content 
(what citations and altmetrics are, how they are sourced, and so on). The rest of 
the two-hour workshop was a mix of demonstrations and hands-on activities with 
tools like Web of Science and Google Scholar. We wrapped with a Q&A panel that 
included two librarians and the Vice Chancellor of Faculty Affairs. 

We learned two major things in the first workshop: 

1. Many faculty already had Google Scholar profiles, and 

2. Faculty were more interested than we assumed in altmetrics. 

There was enough interest expressed post-workshop that we expanded the 
altmetrics section of our next workshop. Around that time, we also began offering 
this workshop each fall and spring semester, rather than once a year. 

This allowed us to differentiate the focus of the workshops a little each time: 
for example, one year, we held two workshops (one for health professionals, 
science, and technology and another for humanities); another year, we focused on 
demonstrating impact in public scholarship and civic engagement, as well as for 
team science. Our most recent workshops have focused primarily on altmetrics. 
We have also begun to differentiate workshops and guidance for the types of 
products and scholarship that faculty across campus are creating. 

In general, the content covered in these workshops includes the following: 

• Why metrics - proxy for quality 

• Types of metrics 

Journal-level, author-level, article-level citation metrics 
Web and social media metrics 

• Sources of metrics 

Aggregators 

Publishers 

Repositories 

• Evaluating metrics (check out this comparison table I created) 

• Tips for gathering data 

• Strategize 

Gather 

Record 

Select 

Visualize 
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The workshops have been so successful because we brought librarian expertise 
to a support system that was already established and in demand. Had we tried to 
host these workshops on our own, they would not have been as well-attended. 

The most valuable aspect of this process for me has been the collaborative 
relationship developed with the Office of Academic Affairs (OAA). Thanks to 
them, I have learned a lot about the promotion and tenure process at IUPUI, as 
well as the dynamics between campus-level P&T guidelines and department- and 
school-level guidelines. 

My advice for building these collaborative relationships with units like Academic 
Affairs is to go slowly and focus on developing a constructive dialog. Like the 
library, they exist to support faculty. Building a network of support in which faculty 
can succeed is a major driver for this collaboration. It also helps to counter the 
perception that libraries exist only to circulate books. 

[ Editor's note: More tips for planning successful workshops can be found in the 
"Planning Altmetrics Workshop" chapter.] 

Being change agents 

Advocating for the use of diverse metrics for impact and reputation in P&T requires 
engaging with faculty, departmental administration, and campus administration to 
make change on campus. While “change” was not necessarily a goal when we 
started offering these workshops, guerilla advocacy has become a part of my 
conversations with faculty. 

One advocacy tactic is to help researchers step back to see the value of all their 
scholarly products in new ways. An easy way to get them to think outside the 
box is to have them list all of the products resulting from a specific research 
project. This usually includes presentations, posters, white papers, policy reports. 
Sometimes faculty list code, models, data, and teaching materials, depending on 
their discipline. 

With these specific products in mind, I then describe a couple of scenarios for 
how altmetrics can showcase the attention for individual items. This helps them to 
understand the power of altmetrics: rather than relying on a metric that describes 
the impact of the container for their work (i.e., the journal), they can point to specific 
evidence for exactly how their presentation or blog post or syllabus has been reused. 
This type of evidence is also more powerful for supporting a P&T case because it can 
relate directly to the items produced during the period of review. 

The promotion and tenure process is about demonstrating potential for contributing 
to scholarly knowledge in your discipline, but that’s difficult to do in the relatively 
brief window that faculty have to publish before they go up for tenure. Given the 
lag in accumulating citations, a great way to get buy in for altmetrics is to help 
faculty understand the portfolio of metrics that they can use to demonstrate the 
impact of their work in multiple areas, more quickly than citations. 
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You can also raise awareness of altmetrics more generally through service on 
campus committees, getting involved with new faculty orientation, and engaging 
department chairs in discussion of their priorities and criteria. As a champion 
of altmetrics, the most important step is to become an adopter yourself - use 
altmetrics in your own promotion and tenure materials and discuss their value with 
your library’s promotion and tenure committee. 

Become an altmetrics expert 

There's no better way to be an effective instructor than to know that topic firsthand. 
So, you should become a user of altmetrics before you offer a workshop on them. 
Try to use altmetrics for your own professional advancement (annual reviews, 
promotion & tenure, grant applications, etc)- in doing so, you’ll very quickly learn 
the best places to find altmetrics data and which data types are the most useful to 
demonstrate particular flavors of impact. 

I am usually my own guinea pig for the strategies and tools I recommend in our 
workshops. I do my best to walk the talk, so to speak. In my mid-tenure review, I 
included a table of usage statistics (page views and downloads) for my materials in 
lUPUI’s institutional repository and on Slideshare, plus the full text of a few tweets 
related to my conference presentations. In my last annual review, I included a 
screenshot of my Impactstory profile, Storify conversations about my conference 
presentations, and an extensive table of metrics for my presentations on Slideshare. 

My dossier is due in May 2016. My case will be based strongly on my engagement 
with and contribution to data librarianship as demonstrated by altmetrics, at least 
primarily. I am fairly confident that the reviewers will see the value of this evidence, 

at least at my own institution. 

Since preparing for my mid-tenure review (where I collected most of my impact data 
manually), aggregation tools like Impactstory, the Altmetric bookmarklet, Google 
Scholar, and PlumX (a subscription database) have made gathering impact data 
much simpler. These services collect data from across the web and incorporate it 
into a single, article-level or researcher-level report - give them a go for yourself! 

One final way to be an altmetrics expert is to keep on top of the altmetrics 
research literature. New studies are published all the time and are often shared in 

this Mendeley altmetrics group. 

To get started, check out these resources: 

• Altmetrics: A Manifesto 

• The Right Metrics for Generation Open [Impactstory blog] 

• Seven ways to make your Google Scholar Profile better [Impactstory blog] 

• Altmetrics in the wild: Using social media to explore scholarly impact 

[Arxiv preprint] 
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• Altmetrics: What, why, and where? [ASIS Bulletin] 

• he power of altmetrics on a CV [ASIS Bulletin] 

• Ten ways librarians can support altmetrics [Altmetric blog] 

These readings contain great content (including strategies for using altmetrics and 
examples of researchers who have used altmetrics for grants and tenure) that you 
can borrow from when creating workshops. 

• Steve Roberts' Lab - Products page integrating Altmetrics 

• Erin McKiernan - Right to Research Workshop [Slideshare presentation] 

• Fernando Maestre - How I use altmetrics [blog] 


Webpages and LibGuides 

Beyond in-person outreach, many librarians opt to create library webpages or 
LibGuides that explain altmetrics and other research metrics. 

One recent study of this “passive outreach” strategy found that sixty-one of sixty- 
two libraries belonging to Association of American University member-institutions 
offered such documentation. These websites tended to contain the following types 
of metrics: 

• Journal Citation Reports (100%) 

• Journal Impact Factor (98.4%) 

• H-index (90.2%) 

• Altmetrics (80.3%) 

Other issues discussed included tools for finding various metrics and researcher 
identification systems like ORCID. 

If you’re interested in creating a LibGuide on altmetrics at your institution, we 
recommend checking out these up-to-date, balanced LibGuides first. 

• “Altmetrics”, University of Melbourne Library: Solid introduction to altmetrics 
and their benefits & limitations. Includes a list of tips on how to improve your 
altmetrics through social media outreach. 

• “Research Impact, Citation Analysis & Altmetrics”, University of Cincinnati 
Health Sciences Library: A comprehensive guide to traditional citation metrics 
and altmetrics. Includes guides and tutorials on how to find citations for your 
work, and (to a lesser extent) information on altmetrics tools. 

• “Traditional & altmetrics” [page], Utrecht University Library: Part of a larger 
primer on research impact and visibility. Includes a comprehensive grid that 
compares metrics offered by and features of altmetrics services and citation 
databases. 
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• “DataBank: Scholarly Metrics intro 11 , Northwestern University Library: 
Information on scholarly metrics (including altmetrics) is broken down 
according to what’s being assessed (articles, journals, databases, etc). 

• “Iowa State University Guide to Altmetrics,” Emma Molls & Megan O’Donnell, 
Iowa State University: A concise guide to what altmetrics are, where they come 
from, and how researchers can use them. 

• “Altmetrics”, Andrea Horne Denton & Kimberly Barker, University of Virginia 
Health System Moore Library: A solid overview to altmetrics, aimed at 
beginners. Includes suggested readings and other educational resources. Useful 
for librarians. 

• “Altmetrics, las otras metricas: <LQue es altmetrics?”, Biblioteca Universidad 
D Sevilla: An excellent Spanish-language primer on altmetrics, filled with 
references to other Spanish-language altmetrics research and resources. 

You are also welcome to adapt and reuse content from WhatAreAltmetrics.com. It’s 
available for reuse and remixing under a CC-BY 4.0 license (please give authorship 
credit to Altmetric). 

For more information on how to copy existing Libguides and Libguide pages into 
your own Libguide, see this tutorial. 


Encourage the use of altmetrics tools 


Usually, having a subscription to a database is not enough— you need to let your researchers know 
that they have access and help them learn how to use it, too! 

The University of Stirling has created a comprehensive LibGuide to accompany their Altmetric 
Explorer for Institutions subscription. It includes: 

• Easy-to-understand documentation for Altmetric data 

• Step-by-step guides for using the Altmetric Explorer for Institutions database 

• Targeted guides for and students on how to use Explorer for Institutions to achieve 

specific goals, like learning when their research is being discussed or finding “trending” research 
in their discipline. 

If you have a subscription to an altmetrics tool at your institution (or want to encourage the use of 
free tools like Impactstory profiles or the Altmetric bookmarklet), creating a similar guide is a good 
way to go! 
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Workshops & brown-bags 

Following are examples of presentations that have been created by librarians 
worldwide to explain altmetrics to both faculty and their fellow librarians. You may 
find these examples to be useful starting points for thinking about creating your own 
altmetrics presentations. For an in-depth guide to workshops that goes beyond the 
basics, check out the “Planning altmetrics workshops’’ chapter that follows. 

Researcher-oriented workshops 

Many universities host workshops for faculty and graduate students on altmetrics and 
other research metrics. Here, we’ve collected some of the very best for your reference. 

• “10 Easy Steps to Open Scholarship”, Alison Hicks & Andrew Johnson, 
University of Colorado Boulder, 2014 

• “Enhancing the visibility and impact of your research”, Karen Gutzman, 
Northwestern University, 2014 

• “Altmetrics for research: impact measurement & #hcsm”, Robin Featherstone, 
Alberta Research Centre for Health Evidence, 2014 

• “Social media and altmetrics for scientists”, Wouter Gerritsma, Wageningen 
UR Library, 2014 

For a “meta” view on designing a library workshop on altmetrics, see Young-Joo Lee’s 
poster, which describes how she conceptualized her workshop’s design-from needs 
assessment to delivery to outcome measurement. 

Librarian-oriented workshops 

Some libraries also offer training for librarians on altmetrics and bibliometrics. The 
following presentations may be helpful in planning a similar training session at your library. 

• Introduction to altmetrics”, Linda Galloway et al, Syracuse University, 2013 

• “Measuring research impact with bibliometrics”, Tanya Williamson, Lancaster 
University, 2014 

• “Altmetrics, Apples & Oranges”, Pat Loria, University of Western Sydney, 2014 

• “Joining the ‘buzz’: the role of social media in raising research visibility”, 

Eileen Shepherd, Rhodes University (now retired), 2014 

• “Are the scientists onto something with altmetrics?”, Katie Brown, Charlotte 
School of Law, 2014 


Altmetrics for Librarians: Outreach on Altmetrics: A Complete Guide 


15 


General Overview 


Talking points 

Anyone who introduces the concepts of altmetrics often hears a number of common 
questions and concerns about this new type of data. Below, we’ve collected talking 
points and answers to frequently asked questions that you can use to practice discussing 
altmetrics in one-on-one conversations with your fellow librarians and faculty. 

Altmetrics are complements for traditional metrics, not replacements: Altmetrics 
have several advantages over citations: they’re quicker to accumulate, they document 
non-scholarly attention and influence, and they can be used to track the attention 
for non-traditional research outputs. However, they can’t tell us anything about 
the quality of the research. Citations are a useful indication of traditional scholarly 
influence, whereas altmetrics can tell us about public influence and non-traditional 
scholarly influence, which can occasionally predict later citations. You need both 
kinds of metrics to get the full picture of research’s value. 

A majority of research doesn’t have attention online— and that’s OK: It’s likely just 
not being discussed on the sites that altmetrics aggregators track, like more than 80 % 
of other research online. 

Altmetrics are great for tracking the attention for datasets, software packages, 
and other non- traditional research outputs. Altmetrics can prove just how valuable 
research software, data, and other outputs are to other scholars, policymakers, and 
members of the public. For example, take the metrics for this script; Depsy shows that 
hundreds of other people worldwide need it run their own software. That’s influential! 
And now its creators can get the credit they deserve. 

Altmetrics can also help authors learn how their traditional research (articles and books) 
are making a difference outside of academia. A few ways authors can see this include: 

• Measuring the influence of research upon climate change public policy 

• Understanding public reception for history research via Goodreads ratings 

• Discussion in “trade” books, as understood via Google Books citations 

Altmetrics can show us scholarly influence beyond citations: Researchers do more 
than just cite articles: they save books to their reference libraries (creating Mendeley 
and CiteULike readership numbers); they adapt others’ data (GitHub forks) and use 
it in their own analyses (data citations); and so on. Altmetrics are traces of these 
“alternative” scholarly activities. 

The value of altmetrics is primarily in the qualitative data that they can surface: It’s 
widely acknowledged that citation counts and usage data can’t tell a person much 
about how research is actually used and regarded. Raw metrics can’t distinguish 
between a citation that celebrates research and a citation that refutes another article. 
They also can’t distinguish between the number of Mendeley readers that save 
an article versus those who go on to read it and cite it. However, the underlying 
qualitative data can serve as a signal for potential impact. Think of metrics as the 
smoke that signals the potential for (impact) fire. While metrics in isolation can be 
good indicators of simple attention, you need the qualitative data to truly understand 
research’s impacts. 


Altmetrics for Librarians: Outreach on Altmetrics: A Complete Guide 


16 


General Overview 


Frequently Asked Questions 

How are altmetrics data collected? Though the specific mechanics of how altmetrics 
are collected differ between providers, in general the process works like this: 

1. The altmetrics service (Altmetric, PlumX, Impactstory, etc) learns of a new piece 
of research: either the work is published in a journal that the service tracks, an 
end user adds a work to their profile on the service, or a third-party like ORCID or 
CrossRef adds a record of the work to their system and then notifies the altmetrics 
service. 

2 . Using a unique identifier associated with the work (e.g. a DOI, PubMed ID, 
URL, or other character string), the altmetrics service searches the data 
sources they track (e.g. Twitter, Wikipedia, policy documents, etc) for 
mentions of the work. Sometimes, the altmetrics service will supplement this 
practice by also searching the source for author names or publication titles. 

In certain cases, PlumX queries the platform that hosts the work to gather 
platform-provided metrics (e.g. article downloads and views on a publisher’s 
site, comments on YouTube, etc). 

3 . The altmetrics service then searches its own database to find similar versions 
of the work and to compile the metrics for each version into a single report. 

This practice is especially useful in instances when a journal article and a 
preprint of that article are both discussed online. 

Altmetrics, ALMs, usage statistics, social media metrics: what’s the difference? 

• Altmetrics are any data sourced from the Internet that tells us who is discussing 
research online, how often they’re doing it, and in what contexts. News stories, 
policy citations and peer reviews are all types of altmetrics, but some have 
argued that it would be useful to group altmetrics under different headings, as 
they can tell us about different types of influence. 

• Usage statistics are downloads, page views, session length, and other server log 
data that can tell us how many people visited an output online and how often 
they visited it. 

• Article-level metrics are any metrics for a piece of research at the article (or 
output) level. They can include altmetrics, usage data, and even citations, but 
they shouldn’t be confused with aggregated measures of attention such as the 
Journal Impact Factor and h-index. 

• The above metrics are all distinct from social media metrics , which can include 
Facebook likes or shares, the number of followers a researcher has, or the 
number of views for a YouTube video that’s unrelated to a specific piece of 
research. Altmetrics generally track metrics (including social media metrics) 
related to discrete research outputs; social media metrics relate to pretty much 
anything else. 
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Aren’t altmetrics easy to game? The best altmetrics tools tend to a) employ anti- 
gaming strategies to avoid artificially inflated metrics, and b) downplay the quantitative 
data and emphasize the qualitative data (making it much easier to identify instances 
of gaming). Altmetric, Plum Analytics, and PLOS all use means to limit the effects of 
gaming and self-promotion. 

What about negative tweets/citations? They’re not a big of a problem as you might 
assume. Research has shown that only a minority of tweets (0.9%) and citations 
(5-14%) are negative. It’s more common for a Twitter user to share an article’s link 
alongside a relevant hashtag, and citations are more often "perfunctory” in nature 
than they are negative. 

Aren’t altmetrics just social media counts? Social media makes up only a small part 
of altmetrics. Citations to research in policy documents, post-publication peer reviews, 
and expert recommendations in Faculty of 1000 Prime are also types of altmetrics. 

How can you possibly measure impact on clinical practice? This can be done by 
understanding how practitioners put research into practice. Proxy data for understanding 
that include: readership on PubMed Central (where many practitioners access research) 
and citations to research in public health policy documents, manuals, and more (where 
recommendations for health interventions, protocols, and so on are often made). 

What about impact on public policy? That’s possible to track via citations to research 
that appear in public policy documents. Altmetric tracks these sorts of citations from 
a number of government and NGO policy bodies. 

What about impact on education? Citations to research in syllabi is just one type 
of altmetric that can inform research’s influence upon education. Altmetric recently 
started tracking mentions in syllabi via an integration with the Open Syllabus Project; 

learn more here. 

When will there be standards for altmetrics? The National Information Standards 
Organization (NISO) is currently working on a set of standards and best practices 
for altmetrics. For more information and for an up-to-date timeline on when these 
standards will be released, visit their website. 

You can’t quantify scientific quality; many great papers are not highly cited. We 

agree that quality is best judged by reading scholarship and making a personal 
judgment call. But there are a number of well-respected expert peer review sites 
(including Faculty of 1000 Prime, Pubpeer, and Publons) that crowdsource such 
judgments. Those sites can be mined for altmetrics. 

What are tenure and promotion committees doing about altmetrics? A small but 
growing number of universities and departments are beginning to include altmetrics 
in their tenure & promotion dossier preparation guidelines as one example of the 
many types of data that researchers can use to document their influence. And even 
at universities that do not explicitly mention altmetrics in their guidelines, researchers 
are successfully documenting their influence using altmetrics. 
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Do you have any good examples of altmetrics making a difference? Yup! Check out 

this researcher-focused Altmetric blog post for some great examples of how data- 
savvy researchers have used the data to monitor and showcase the conversations 
around their research. 

Chapter Summary 

• Building grassroots support via outreach to fellow librarians and faculty is a 
crucial component to getting buy-in for altmetrics-related programming and 
services on your campus. 

• Faculty, students, and librarians are three distinct communities; your outreach 
tactics should be specially tailored to each for maximum success. 

• Building campus-specific reference resources like LibGuides can help you 
connect with those on your campus interested in altmetrics. The best reference 
resources include a number of specific points. There are a number of pre- 
existing resources that are licensed in such a way that you can freely reuse that 
content, saving you time and effort. 

• There are often common concerns that those new to altmetrics have. Practicing 
talking points and your answers to frequently asked questions will help you be 
prepared for most one-on-one discussions you’ll have regarding altmetrics. 
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Planning Altmetrics Workshops 

In this chapter, well share some general tips for planning great altmetrics workshops, 
including content you can repurpose so you’re not starting from scratch. 

In the last few years, we’ve organized dozens of researcher-oriented workshops on 
altmetrics. 


Beyond that, we’ve talked to hundreds of researchers and librarians who have 
organized their own successful altmetrics outreach events. 

The common theme from everyone’s experience? 

Running a workshop is hard. Getting people to come to a workshop is harder. But 
both are totally doable. 

In this chapter, we’re going to share some tricks and resources that can help you plan 
your own perfect workshop on altmetrics. By following these tips, you’ll get more 
attendees, save yourself time, and earn increased recognition as an altmetrics expert 
on your campus. 


Tip 1: Learn the rules for creating engaging 
presentations 

Until very recently, we had assumed two things: that some people were just inherently 
good at designing slides, and that some people were born with the ability to captivate 
a room during presentations. We were wrong on both points! 

PowerPoint design is something that can definitely be learned. There are a number of 
resources available to help you learn the basic rules. 

Short on time? A nice, quick hack we’ve learned is to find a “visual precedent.” (This is a 
polite way of saying, “biting someone’s style.”) Some of our favorite visual precedents 
tend to come from librarian Brianna Marshall’s presentations. We flip through her 
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slide decks and decide what we like about them, then sometimes use those design 
elements in our own presentations. 

For example, we love Brianna’s use of bold images and very little text: 



the lifeblood of an R1 institution! 


Figure 1. A slide from one of Brianna Marshall’s presentations 
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We've adapted those elements into one of our own presentations: 



Figure 2. A slide from Stacy Konkiel’s presentation that borrows from Brianna’s “visual precedent” 


If you’ve ever seen a colleague’s presentation and thought, “Oooh, shiny!”, then you 
may wish to use their slides as your own visual precedent. 

Creating a slick slide deck is only half the battle, however. You also need to work on 
your speaking skills, so the presentations you give will keep your audience hanging 
on your every word. 

Sound hyperbolic? Well, we give presentations on research impact metrics for a living 
and-while we’re by no means the best speakers on Earth-we’ve been told that we 
make following a dry topic very enjoyable for the listener. 

Public speaking is a set of skills that can be learned like any other, and there’s 
actually a science to creating a presentation that will hold your audience’s 
interest. Check out this surprisingly thorough infographic "cheat sheet” created by 
LondonSpeakersBureau.com for loads of tips, including how to build a compelling 
story into your talk. 

Resources: 

• Unsplash - high-resolution images to use in your next presentation 

• Flickr Creative Commons search - more presentation-ready images 

• The Charisma Myth - a great book on connecting with others, including 
audiences 

• Public Speaking for Normal People - if you’ve only got 5 minutes to spend 
improving your speaking skill set, read these rules 
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Tip 2: Think “what’s in it for me?” 

Want your audience to care about what you say? Put yourself in their shoes for a 
moment and think about what’s most important to them. Then-and only then-should 
you start crafting your workshop talk. 

For example, we can guarantee that the average researcher probably obsesses over 
at least one of the following topics: 

• How to get a job (especially if they’re a student or early career researcher) 

• How to get tenure 

• How to get grant funding (especially if they are a scientist) 

• How to find collaboration opportunities with other researchers 

• What other people are saying about their research 

You’ll notice a theme: the first four points are all about career advancement, and the 
last point relates to the basic human instinct of vanity. 

Find ways to build your presentation around just one of those points. It’s too much 
ground to cover otherwise! 

When you’ve decided upon your central theme, start collecting altmetrics resources 
and examples that relate to that theme. You’ll want to craft a compelling story related 
to what each researcher’s most concerned about. Here are some to get you started: 

• Job applications: Find job announcements in the discipline(s) that you are 
targeting for your workshop, then mockup what an altmetrics-boosted cover 
letter or CV would look like. You might also feature a quote from a department 
chair at your university about the importance of showcasing diverse impacts in 
a job application. 

• Tenure: A bunch of examples of the use of altmetrics in promotion and tenure 
dossiers can be found on this LibGuide from Duke. Use these examples as 
starting points to create a story. (“This is Ahmed. Here’s how he used altmetrics 
in his tenure package to prove that he’s developed influential research 
software.”) 

• Grant funding: Similarly, you can create stories based on examples like C. Titus 
Brown and other researchers who have used altmetrics to secure grant funding. 
Be sure to also speak to the specific requirements of funders like the Wellcome 
Trust, NIH and NSF, who all have “broader impact” mandates. Altmetrics are 
great for discovering broader impacts. 

• Collaboration opportunities: Some researchers use altmetrics for “serendipity”- 
finding other researchers who discuss their work and its implications on social 
media, then reaching out to explore joint research projects, grantmaking, and 
more. Here are some examples of discussions that could result in collaboration 
opportunities from astronomy, virology, and climate change studies. 
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• What other people are saying: Ask attendees to name a popular research 
article or book that they’ve recently read, find it online, then look up altmetrics 
for it using the Altmetric bookmarklet or Impactstory. Then, allow attendees 
time to experiment using those altmetrics tools to look up conversations around 
their own articles and books. 

One more bit of advice: don’t treat your researchers as if they’re all the same. Senior 
faculty will have different career concerns from early career researchers; humanities 
scholars will see things a bit different from their colleagues in the sciences. Try to 
tailor your workshops based upon those differences-it will make the information you 
share much more useful for your attendees. 

Tip 3: Leverage other people’s awesomeness 


Using Altmetrics 

What is your case? What statements do you need to 

support? 

• Identify priorities 

• Choose tools/platforms 

• Gather 

• Select 

• Store 

• Visualize 

MarchS, 20 IS 


Figure 3. A slide from “Altmetrics for Team Science” by Coates & Miller (2015) 

There are loads of great altmetrics presentations floating around on Slideshare and 
Figshare. Why reinvent the wheel when you can borrow from them? (Giving full credit, 
of course!) 
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Here are some presentations on altmetrics that are chock-full of useful information: 

• “Altmetrics" by Micah Vandegrift (FSU): An informative look at how altmetrics 
and citation metrics are complementary and showcase different kinds of 
influence. 

• “Altmetrics for Team Science” by Heather Coates & Willie Miller (IUPUI): A solid 
description of how large research teams can better approximate their value 
using a variety of metrics. 

• “Social Media in Science and Altmetrics - New Ways of Measuring Research 

Impact” by Christoph Lutz: A science-focused examination of how evolving 
modes of scholarly communication are changing the way we understand 
impact and measure attention. 

Librarian Heather Coates (IUPUI) recently shared her thoughts on themes to emphasize 
in altmetrics presentations. The themes include: 

• The research cycle doesn’t end with publication 

• Your scholarship is more than just publications 

• You are the product (i.e. “Demonstrating the impact of your work is mostly 
useful in communicating that you are a high-quality, productive scholar 
contributing to the missions of your institution, school, and department.”) 

• Have a plan - strategically plan how you will disseminate your work 

• Execute the plan - treat the work of disseminating, sharing, and tracking 
evaluation of your scholarly products as a project 

• Review and thoughtfully select the evidence supporting your argument - 
demonstrate that you are a productive scholar/teacher/practitioner worth 
keeping around 

Want other ideas on how to frame your discussion? Check out the altmetrics talking 
points in the “Outreach on altmetrics: a complete guide” chapter. 
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Tip 4: Know your stuff 

Researchers are a notoriously skeptical bunch. (After all, a big part of the peer review 
process lies in picking apart others’ arguments!) If you don’t speak with confidence 
about altmetrics, your audience will tune you out and-by association-will start tuning 
out altmetrics discussions elsewhere, as well. 

The first step to “knowing your stuff” is to become deeply acquainted with the 
culture at your institution and within its various departments. Talk to your colleagues 
within other departments and, if possible, to department chairs to understand: Are 
researchers encouraged to do outreach via social media? Is Open Access publishing 
a priority? How are researchers evaluated for tenure and promotion, or during their 
annual reviews? 

Answers to all of these questions will help you understand how seriously researchers 
will take altmetrics, and that can help you tailor your workshop message. 

We’d also recommend including research-backed insights from a variety of unbiased 
sources to back up your claims that Mendeley readers correlate with later citations 
or that humanities research is better covered by altmetrics databases than citation 
databases. Other studies to point to include Heather Piwowar et al’s theory of "flavors 
of impact” and this study that shows that Open Access research articles get more 
downloads, citations, and attention on social media. More altmetrics research can be 
found in this "bibliography”. 

Finally, you should be prepared for some hard questions. Senior faculty and librarians 
can be especially skeptical of altmetrics, and they may grill you during the workshop 
Q&A session. 

Practice your answers to these questions to be one step ahead of the Doubting 
Thomases: 

• Why do you want to replace citations with tweets? You don’t! Instead, you 
recognize how powerful these complementary metrics are. 

• Won’t altmetrics just incentivize sensationalism in science? Not at all! The 
beauty of altmetrics is that they can capture important non-scholarly value like 
influence upon policy, technology commercialization, and education. Altmetrics 
are NOT just about “buzz”-that’s a big misconception that many people have! 

• Does one have to be active on social media for altmetrics to work? Because 
I think Twitter/Facebook/XYZ is a waste of time: Luckily not! Whether or not 
you believe in the power of social media, other people do-and they’re using it to 
discuss, debate, recommend, and save references to research every day. 
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Tip 5: Know your audience 



Figure 4. Image used under CC-BY license-Apps for Europe / Flickr 


Here are some tips for reaching specific demographics for your workshop: 

• Use (well-designed) posters: Given email overload, many librarians are 
resorting to the tactic of advertising their events using good old-fashioned print 
posters, hung in faculty and graduate student lounges. It’s easy to delete an 
email; it’s more difficult to ignore a well-designed poster hung over the faculty 
lounge microwave as you wait for your food to cook. 

• Offer food: It’s a fact that graduate student-oriented events are 1500% more 
successful if food is provided. And who among us-student or not-could ever 
turn down a nice hot cuppa and snacks offered during a useful event that 
doubles as a respite from grading student work? Food is a nice way to get 
people in the door. Be sure to advertise your planned refreshments prominently. 

• Schedule with your audience in mind: Trying to reach post-docs in the life 
sciences? Take care not to schedule your event when a department-wide 
meeting about budget cuts is happening. Trying to reach any tenured faculty 
from any discipline? Don’t schedule during the summer months, when they’re 
likely to be in Barbados on ho l iday doing field work. You get the idea. 

• Don’t be sad if faculty don’t show up: It’s an unfortunate fact that there simply 
aren’t enough hours in the day for the average faculty member to attend all 
the events that interest them. Most have too much on their plate to make time 
for anything that’s non-essential. Offer to record your session and share it with 
registrants after the fact-that way, faculty can tune-in when convenient for 
them, and you’ll still be able to share your message with an interested party. 
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Moreover, consider how research done at your institution (or in your country) may differ 
from other research that’s often discussed in the context of altmetrics. For example, 
Serbian librarians Milica Sevkusic and Zorica Jankovic ran a workshop for fellow 
librarians, where they covered not only the basics of altmetrics, but also examined data 
specifically for research Serbian-authored articles (uncovering the fact that it does not 
receive as much attention as research from other countries, due to a variety of factors). 
Researchers in the UK, on the other hand, would be very interested to know how 
altmetrics might be used in gathering evidence for the REF 2020. Context is important 
and will have an effect upon faculty’s receptiveness to altmetrics. 

Tip 6: Make taking action an irresistible idea 

We probably should have led with this prompt, because you’ll want to plan workshop 
with an actionable outcome in mind. What do you want your researchers to do after 
they leave your event? 

If you want researchers to use metrics appropriately in their next annual review, give 
them handouts or send them a link to a pre-created LibGuide that helps them do just 
that. Your guide should be specific to the annual review process and systems at your 
institution: describe precisely where metrics should go and point your researchers to 
altmetrics tools like Impactstory profiles, the Altmetric bookmarklet, or Scopus item 
records-all are good places to begin hunting for relevant influence and attention metrics. 

If documenting value in a grant application using metrics is your aim, create a handout 
with examples, tips, and tools, or simply point your researchers in the direction of the 
Altmetric “23 diverse metrics to use in your next grant application” blog post. 

If all you want is for researchers to figure out what altmetrics might mean to them and 
contact you if they need help, arm them with a handout or LibGuide described above, 
and also give them your business card with a simple call to action printed clearly upon 
it: “Get in touch for a metrics consultation”. They’ll likely hang onto your business card 
and, when they get around to exploring altmetrics, use it to contact you for help. 

Chapter summary 

• Key to planning a great workshop are messages that resonate with your 
audience and a generous use of eye-catching visuals. 

• Preparation in the form of knowing altmetrics research and anticipating some 
commonly asked questions can go a long way towards making your workshop 
a success. 

• At the conclusion of your workshop, you should always include a “call to action” 
that will inspire attendees to learn how to use and apply altmetrics. 
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A Beginner’s Guide to 
Using Altmetrics for Promotion 
and Tenure 

As a librarian, you may be called upon to support researchers in assembling public 
engagement evidence to be included in their promotion and tenure (P&T) applications. 
An emerging source of such evidence are altmetrics. 

When contacted by researchers for help, it is essential for librarians to offer advice and 
support to help researchers access and contextualise altmetrics data to add relevant 
public engagement evidence to their applications. 

This chapter discusses how to collect and report altmetrics data in promotion and 
tenure dossiers for both researchers and librarians. 

A key point to note up front: academia is still in the early stages of using altmetrics 
in promotion and tenure applications. This chapter adapts expert recommendations 
borrowed from the bibliometrics world and includes known examples of how some 
researchers have included altmetrics in their own promotion and tenure dossiers. 
We also include advice from librarians who are at the forefront of advocating for 
altmetrics in the P&T process at their own institutions. 
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Introduction to the Promotion & Tenure process 

While promotion and tenure requirements vary by university and often even by 
department, the following items are likely to be required in any P&T dossier. We've 
highlighted in bold the dossier sections best suited for the inclusion of altmetrics data: 

1. CV 

2. Personal Statement 

3. Student education and evaluations 

4. Scholarship Evaluation 

5. Teaching Portfolio 

6. Academic leadership 

7. Letters of reference 

8. Service Portfolio 

9. Grants 

Altmetrics can be useful for documenting impact and engagement in P&T dossiers, 
but using altmetrics requires more than just adding raw numbers next to a list of 
papers. Context, meaningful metrics, and specific, full-text examples of influence 
are often necessary in such dossiers. We describe altmetrics’ application within the 
highlighted dossier sections, below. 

Curriculum Vitae 

As we can see above, an up-to-date Curriculum Vitae will be included in any P&T 
dossier. One way to provide a snapshot of attention for individual articles, datasets, 
and other outputs is to encourage researchers to add a static version of the Altmetric 
“donut” badge or the Plum Print next to each line item in their CV (which we explain 
how to do below). By using a simple, static version of these visualizations and metrics, 
you are ensuring that the altmetrics visualisations work across different formats and 
don’t require the reviewer to follow a hyperlink. 


JAMES GRECIAN 


ABOUT/ RESEARCH/ PUBLICATIONS/ PHOTOGRAPHY/ 


Google 

schtMiO 



© 



Grecian W. J„ Taylor G. A.. Loh, G„ McGill, R. A R.. Miskelly, C. M.. Phillips, R. A.. Thompson, D. R. Web / PDF 
& Furness, R. W. (2016) Contrasting migratory responses of two closely-related seabirds to long- 
term climate change. Marine Ecology Progress Series 

Grecian W. J.. Witt, M. J., Attrill, M. 1. Bearhop, S., Becker, P. H.. Egevang, C., Furness, R. W.. Web / PDF 

Godley, B. J., Gonzalez-Solfs, J., Gr^millet, D„ Kopp, M„ Lescro6l, A., Matthiopoulos, J„ Patrick S. 

C.. Peter, H-U., Phillips, R. A., Stenhouse, I. J. & Votier, S. C (2016) Seabird diversity hotspot linked 
to ocean productivity in the Canary Current Large Marine Ecosystem. Biology Letters 
12 : 20160024 



Figure 5. 

A screenshot 
of the use of 
an Altmetric 
badge on James 
Grecian’s online 
CV. The badge 
could similarly be 
included in a print 
P&T dossier CV. 
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Another way that researchers can include altmetrics in their CV is to simply copy and 
paste the data into their dossier. This offers the advantage of integrating the data into 
the existing look and feel of the CV, and also “allows” researchers to retain context 
for their metrics (an important concept that we explain in more detail below). For 
example, Dr. Trevor Branch, a researcher from the University of Washington, copied 
and pasted altmetrics data into his CV to highlight the reach of his publications in the 
media (Figure 6). 


2014 ( 7 ) 

59. Monnahan 2 , C. C., T. A. Branch, and A. E. Punt 1 . 2014. Do ship strikes threaten the recovery 
of endangered eastern North Pacific blue whales? Marine Mammal Science doi: 

1 0. 1 1 1 1 /mills. 1 2 1 57, 1 9 pages. Highest Altmetric score (309) of any paper published in Marine 
Mammal Science, reflecting widespread media coverage (20% of effort, idea, funding, supervision) 

Figure 6. A screenshot from Dr. Trevor Branch’s tenure dossier CV, which includes contextualized 
Altmetric data 


Personal Statement (aka the Tenure Narrative) 

A promotion and tenure dossier often contains a personal statement or tenure narrative, 
which allows your researcher to outline of their important academic contributions 
and their service to the university. The Personal Statement provides an opportunity 
to dig down into qualitative altmetrics data for research. For example, facts like as 
“My research was cited in a World Bank report on Malaria" can be discovered using 
altmetrics tools and included in this dossier section. 

Scholarship Evaluation 

Scholarship evaluation is often a required section in promotion and tenure dossiers; 
essentially, this is the presentation and evaluation of your researcher’s most important 
work. It is your researcher’s opportunity to show they have mastered their research area 
and are leaders in their field. This section offers a great opportunity to draw attention to 
the most compelling altmetrics data for your researcher’s work— specifically, the attention 
data that they believe characterises them as leaders in the field. This is also another good 
place for your researcher to include qualitative evidence, rather than metrics. For example, 
"My research has been consistently cited in mainstream media with broad international 
readership such as the BBC and has had positive Publons reviews" is more compelling 
than "My current Altmetric Attention Scores are 241, 118, and 17". 

Grants 

Promotion and tenure dossiers tend to contain information about grants, both funded 
and unfunded. In the grants section, your researchers can use altmetrics to show the 
attention associated with materials that were published following the grant. Previously, a 
researcher might have been asked to provide the Journal Impact Factor for the journal 
that published materials funded by the grant. Researchers may now choose to include 
the altmetrics for the output itself, allowing the work’s altmetrics to complement its 
citation data. Further, if software or any other non-traditional output is generated during 
the funded work, the researcher can highlight any altmetrics that those outputs generate. 
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Outreach and Engagement 

Outreach and engagement is still a relatively rare requirement for promotion & tenure, 
but the list of universities who do require it is quickly growing. Dr. Branch, whose use 
of altmetrics we saw above, works at an institution with such a requirement. 

Universities such as Virginia Tech have also begun including a “multimedia” section 
in the “outreach and public engagement” section of their dossier. In this section, they 
encourage researchers to participate in non-traditional public engagement strategies 
like guest blog posts. The influence of these blog post themselves can be understood 
using altmetrics. 

In discussing the use of altmetrics in a promotion and tenure dossier, Dr. Ernesto 
Priego from City University London reflected that, “If academic departments value 
public engagement and the production of scholarship that is being discussed by 
individuals online, I do believe that they could be used to complement and provide 
further context to an academic portfolio.” Dr. Priego is the editor-in-chief of the Open 
Access journal Comics Grid. In this capacity, Dr. Priego himself has deposited non- 
traditional research materials in Figshare and done outreach upon them. 

Some professional societies, such as The American Sociological Association, are 
recommending that Outreach and Engagement be formally recognized in promotion 
and tenure requirements. In a report published in August 2016, The American 
Sociological Association discusses the importance of the practice, and even uses 
certain types of altmetrics and usage statistics as a means of illustrating the importance 
of engagement. 

How to responsibly use altmetrics data for 
promotion and tenure 

Before we get into the nitty-gritty of where to find altmetrics to include in a dossier, 
let’s talk about some principles for using metrics responsibly. 

While a “high” Altmetric Attention Score (or any other type of altmetrics) can provide 
a useful indicator of attention to research, the underlying conversations and qualitative 
examples of research are what provide context and value to most metrics. Metrics, 
without context, are meaningless to reviewers. Librarians can therefore help equip 
researchers with the tools and skills to uncover and contextualise altmetrics data: 
supporting them in telling a story about successful impact activities in a way that 
goes beyond a single number. 
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There are three rules one should abide by when using altmetrics (or any metrics) in a 
promotion and tenure dossier: 

• Provide context: A number in isolation doesn’t mean much in a CV. It’s difficult 
for a reviewer to understand if that number is good or bad without another 
number to compare it to. For example, how does attention for a journal article 
compare to that for other articles published in the same journal and in the 
same year? Context is often applied in two ways: through the application of 
percentiles and through the use of qualitative data, which often explains the 
specific context in which research has had an influence. We’ll share some 
examples of both momentarily. 

• Use appropriate metrics: What is the real-world outcome of the researcher’s 
work and what evidence do they have to back up those claims? If your 
researcher is asserting that their research has made a significant contribution 
to their discipline and has been a subject of thoughtful discussion, they must 
be capable of proving that with data. Consider the impact of the project they’re 
documenting in their dossier, and seek out data that bolsters those claims. For 
example, if your researchers are claiming impact upon policy, then you can help 
them find where they’ve been cited in public policy documents. 

• Choose longform evidence: This is compelling information about who’s saying 
what about your research and how they are using it. We refer to this type of 
data as “qualitative data” throughout this ebook. 

These rules are adapted from the Leiden Manifesto’s recommendations, and also themes 
that bibliometrics experts have been advocating for many years. The Leiden Manifesto’s 
10 principles can guide your own understanding of the responsible use of altmetrics. 

Examples of researchers including altmetrics in 
promotion and tenure 

Dr. Ahmed Moustafa, an Associate Professor of Bioinformatics and Systems Biology 
at the American University in Cairo, used altmetrics from his Impactstory profile in 
his P&T dossier to showcase the attention associated with his scholarly outputs. In 
addition to these altmetrics, Ahmed highlighted a few interesting media mentions. 
For example, his 2008 paper in Proceedings of the National Academy of Sciences was 
featured in popular media sources, such as Wired. 
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Total Impact 

Below (Figure 5) is a snapshot of the “impact profile” for a subset of my 
publications provided by “ImpactStory”. Not only the number of citations, 
evaluates the “impact” of each publication, but also recommendation of scholars 
via, for example, FI 000, bookmarking on Mendeley or CiteULike, discussions on 
social networks such as Facebook and Twitter. The detailed impact profile for my 
publications (research articles, datasets, and software) is available at 
http://impactstory.org/AhmedMoustafa. 



Figure 5 - ImpactStory (http://impactstory.org/AhmedMoustafa) 


Figure 7. Screenshot from Dr. Ahmed Moustafa’s Promotion and Tenure dossier 

Dr. Steven Roberts, Associate Professor in the School of Aquatic and Fishery Sciences 
at the University of Washington, used altmetrics in his P&T dossier to showcase the 
attention associated with his scholarly outputs and engagement activities. In the 
chart below, you can see how many times Dr. Roberts’ datasets and documents have 
been viewed through his Figshare profile. This attention is bolstered by his own public 
engagement activities that include blogging, tweeting, and depositing his work in an 
Open Access repository. 
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Platform 

Purpose 

URL 

Metrics 

Weosite 

Primary Lab Website 

facurty.washington.edu/sr320 

2 months -218 
unique visitors 

Wiki 

Share online lab notebooks and 
protocols, lab communication 

genefish.wiklspaces.com 

2012 - 20k views 

Blog 

Lab Tumblr 

geneflsh.tumblr.com 

12 months - 6137 
visits 

Facebook 

Shares Lab Tumblr posts 

goo.gl/pOitK 

47 likes; IlkFnends 
of Fans 

Blog 

Blog specifically documenting our Ocean 
Acidification Research 

safsoa.wordpress.com 

2012 - 1645 views 

Twitter 

Automatic lab feed - notebook entries, 
posts, events 

twitter.com/genef1sh 

17.330 Tweets 

Twitter 

Personal twitter account 

twrtter.com/sr320 

1491 tweets 

Rlckr 

Sharing research images 

googl/yzwE7 

2726 photos 

Youtube 

Pnmanly host Instructional videos 

youtube, com/user/srtab 

7351 views 

Scrlbd 

Share documents, pnmanly undergrad 
student papers 

scnbd.com/sr320 

9908 reads 

Figshare 

Shartng datasets and documents 

goo.gl/umGFg 

4947 views 

Sildeshare 

Shanng presentations 

slideshare.net/sr320 

49 sildeshares 


Figure 8. Screenshot from Dr. Steven Roberts’ website, included in his tenure dossier 

How to include evidence in your researcher’s dossier 

Traditional metrics 

The core of any promotion and dossier for many disciplines are journal articles, books, 
and book chapters, and those related outputs’ number of citations. Departments 
might also want to know in which journals researchers have published and, at some 
institutions, the Journal Impact Factor of said journals. 

Often, researchers will have that information readily available on their CV. But in 
some cases, they may need your help to update their CV with their current list of 
publications, as well as the Impact Factors for the journals in which they’ve published. 

There are a variety of helpful guides to finding exporting traditional citation data, so 
we won’t reinvent the wheel here. For help to finding journal impact factors, check out 
this guide by Yale University. To look up a current list your researchers’ publications in 
Scopus, learn from this City University Hong Kong tutorial; a similar guide for searching 
Web of Science is available from the University of Michigan Libraries. 

Qualitative Data 

A solid source of evidence that will help your researcher craft a compelling “impact 
story” (to borrow a useful phrase) can be found in qualitative data: who’s saying what 
about your researcher’s work. 
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Dr. Trevor Brand used qualitative data to document the value of his science outreach 
efforts in his tenure dossier, in particular to showcase examples of online engagement 
he had with high-profile authors like Joyce Carol Oates and data visualization experts 
like Edward Tufte. 

If he were making an argument about the worldwide influence his engagement 
efforts have had, Dr. Brand could have gone one step further by adding Twitter or 
Mendeley maps to his dossier, or highlighting the fact that his work has received a 
lot of interest from other scientists worldwide. These types of data are available on 
Altmetric details pages (under the Summary tab), and freely accessible using the 
Altmetric Bookmarklet. 

If Dr. Branch wanted to include specific examples of policy impacts or mainstream press 
coverage his work has received, he could have also used the Altmetric Bookmarklet 
to easily find links to those types of attention, and he could have copied and pasted 
mentions directly from Altmetrics details pages for his work. 

Qualitative evidence can be found in other altmetrics services, as well. PlumX 
Dashboards and Impactstory also both include some types of qualitative data on 
researchers’ profiles. 

Evidence of impact beyond authorship 

Altmetrics provide an opportunity for researchers to get credit for their work, beyond 
what they’ve happened to write a journal article about: creating and curating data, 
presenting at national and international conferences, coding widely-used software, 
and so on. Highlighting these “non-traditional” outputs can help researchers present 
a more holistic view of the value of their work. 

However, to get altmetrics, research has to be discoverable by others. That means 
putting the stuff that’s normally kept private on one’s Dropbox account up onto the 
public Web. Visualisations, datasets, videos, and all other research outputs can be 
deposited in institutional repositories or in open access repositories like Figshare, 
and— once they’re “live”— can start accumulating altmetrics attention. Try to educate 
researchers on the value of this practice well before the point of preparing for 
promotion & tenure, so they have as much time as possible to capture attention data 
for their work. 

Contextualized Raw Metrics in a CV 

When looking to include raw metrics to your researcher’s CV, it is important to add 
context. The best ways to do so are through the use of badges and percentiles. 
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Badges 

Impactstory is a free researcher profile platform that gathers and displays altmetrics 
for a researcher’s entire body of work, at a glance. 

Impactstory provides context using “Achievement badges". The badges summarize a 
researcher’s influence within four categories: 

1. Buzz-How much is your research being talked about? 

2. Engagement-Who is engaging with your research? 

3. Openness-How easy is it to access and understand the materials? 

4. Fun-Amusing, not serious, perspectives on influence and “impact”! 

To find Achievement badges for your researcher, first create an Impactstory profile for 
them. The profile must be linked to an ORCID ID, which are free to make. Impactstory 
displays all the published outputs that have been included in an ORCID profile and 
then collates attention associated with them. 

Once your researcher’s profile has been created, you can find any relevant badges in 
the “Achievements” tab on main landing page for the profile. You can choose to view 
all the achievements or sort by one of the four dimensions described above. 


7 ACHIEVEMENTS 



Global Reach PI Top 50% 

Your research has been saved and shared in 29 countries. That's high: only 40% 
of researchers get that much international attention. 

O Countries include Australia, Austria, Belgium and 26 more. 



Wikitastic 

Your research is mentioned in 1 Wikipedia articles! 
O Your Wikipedia titles include Altmetrics . 


Filter by dimension 

buzz (l) 

A engagement (4) 

fi openness (i) 

© fun (i) 



Greatest Hit PI Top 50% 

Your top publication has been saved and shared 118 times. Only 42% of 
researchers get this much attention on a publication. 

C? Your greatest hit online is New perspectives on article-level metrics developing 
ways to assess research uptake and impact online. 



Open Access 

60% of your research is free to read online. 



Global South 

Of people who save and share your research, 17% are in the Global South. 
O Countries include Brazil, Chile, India and 8 more. 



Follower Frenzy *to P 50% 

Someone with over 33.976 thousand followers has tweeted your research. 
Only 39% of scholars have been tweeted by someone with this many followers. 
C> Thanks, @MSF uk . 


Figure 9. A Screenshot of different achievement badges from Impactstory 
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Percentiles 

The Impactstory achievement badges contain percentile information: how attention 
for the researcher’s work compares to attention received by all research that 
Impactstory indexes. 

Percentiles are a good way to offer a frame of reference for metrics. They can help 
reviewers interpret the relative influence of research by comparing attention data for 
your researcher’s work with average metrics for other outputs that were published in 
the same year or the same journal. 

Altmetric provides a “score in context” for most of the research it indexes. Using the 
bookmarklet, as described below, a badge will drop down from your toolbar in the 
top right hand side of the window. You can “Click for more details”, which will bring 
you to a details page as seen below. 



Figure 10. A Screenshot of the Altmetrics Details Page for this Article 


From the Summary tab of an Altmetric details page, click on “Attention Score In 
Context”, which will bring up the Altmetric Attention score in various contexts: how 
that article’s Attention Score compares to other articles published in the same journal, 
within the time period (that is, within six weeks of the article’s publication date), and 
compared to scores across the entire Altmetric database. This context is essential to 
an accurate understanding of an output’s attention and relative influence. 


TWITTER DEMOGRAPHICS 


ATTENTION SCORE IN CONTEXT 

V 


This research output has an Altmetric Attention Score of 457. This is our high-level measure of the quality and quantity ol 
Q that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculat 
research output was last mentioned on 20 August 2016. 


ALL RESEARCH OUTPUTS 


#4,927 


of 5,570,954 outputs 


REVIEW LETTERS 

#12 

of 14,014outputs 


r 


OUTPUTS OF SIMILAR AGE 

OUTPUTS OF S 


FROM PHYSIC 

#442 

LETTl 

of 1 33,635 outputs 

#: 


of 153 01 


Altmetric has tracked 5,570,954 research outputs across all sources so far. Compared to these this one has dc 


Figure 11. A Screenshot of the Altmetric Attention Score in Context 
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Where to Find Altimetries Data 

The “Altmetric It!” Bookmarklet is a free tool for 
researchers to use to find altmetrics for specific 
outputs. Here’s how to use it. 

Download the bookmarklet from Altmetric’s website 
by dragging the button onto your browser’s toolbar. 
Then, once you are on the landing page for a journal 
article, you can highlight the DOI with your cursor 
and then click the “Altmetric It” button. A popup, 
seen at left, will appear with a snapshot of attention 
associated with that output. 

You can also generate badges to include in your 
researcher’s CV. Generating an altmetrics badge, 
whether the Altmetric Donut or the Plum Print, is 
very straightforward! 

1. Select a unique identifier, for example a DOI 

2. Go to the Plum Print webpage or the 
Altmetric Donut page 

3. Input the DOI in the Badge Builder 

4. Copy and paste the code into your webpage’s 
source code, or take a screenshot of the Altmetric 
badge or Plum Print to add to your CV 

As mentioned above, Impactstory is a free service that allows researchers to build 
an online profile. The profile must be linked to an ORCID ID, which are free to make. 
Impactstory displays all the published outputs that have been included in an ORCID 
profile and then collates attention associated with them. Impactstory’s achievement 
badges are a great way to find unexpected “flavours of impact” and useful percentiles 
to include in your researcher’s dossier. 

To save time finding altmetrics for your researchers, you might consider using 
subscription altmetrics databases like Altmetric Explorer for Institutions or PlumX 
Dashboards. 

Altmetric Explorer for Institutions is a tool that collects altmetrics for your entire 
institution’s research. It’s searchable by author name, in addition to reporting at the 
department and institution-levels, making it easy to search for and find altmetrics 
for your researcher’s works. You can also use Explorer for Institutions to compare 
attention data for your researcher’s work to that of more than 5 million other works 
we’ve seen mentioned online— a useful way to create ever-important context. 

PlumX Dashboards is another product that allows you to quickly find altmetrics for all of 
your researchers’ work. A bonus of that platform is that they include metrics beyond what 
Impactstory and Altmetric report upon, especially for research outputs like software 
and data. However, the platform does not include percentiles for the metrics it reports. 



■ Picked up by 182 news outlets 
Blogged by 30 
Tweeted by 8232 
Mentioned by 1 peer review 
sites 

I On 190 Facebook pages 
Referenced in 2 Wikipedia 
pages 

Mentioned in 29 Google* posts 
Reddited by 10 
| Highlighted by 1 platforms 
Click for more details 

Figure 11. The Altmetric Bookmarklet 
pop-up image 
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Publishing as a librarian 

At some institutions— particularly in the United States— tenure isn’t just for researchers. 
Librarians may also want to use altmetrics to demonstrate the influence of their work, 
and in fact altmetrics may better serve librarians than traditional faculty. 

Here’s why: many of your outputs as a librarian are likely to be non-traditional and 
outside of the scope of “peer-reviewed journal article”. Posters, conferences, and 
white papers are just some of the ways you might be contributing to the academic 
ecosystem. They should be acknowledged in your promotion and tenure dossier. 
For example, you can archive the tweets about a conference presentation you give, 
share positive discussions of your white paper on research blogs or in a magazine like 
Library Journal, or explain how widely viewed a conference poster has been since 
you’ve archived it online. 

Depositing in an open repository, such as Figshare or an institutional repository, can 
help with some of these issues. Open repositories will mint their own identifiers after 
you’ve deposited the output. For example, Figshare assigns DOIs to all items that are 
deposited within their repository. Some examples of data held within Figshare are 3D 
visualisations, data samples, and posters. Anything that can be deposited is given an 
identifier. Adding an identifier to a non-traditional output allows those materials to be 
shared, garner altmetrics, and allow you to get credit for all of your work, not just the 
stuff you publish about. 

Advocating For the Use of Altmetrics in Promotion 
and Tenure 

Librarians have an important role to play beyond helping researchers find and 
contextualize metrics to include in their dossiers. We are trusted campus information 
experts, and can play a role in advocating for the formal inclusion of altmetrics in 
promotion and tenure preparation and evaluation guidelines. 

Advocating for the inclusion of altmetrics in promotion and tenure preparation and 
evaluation guidelines is easier than you might think. 

You can approach the university administrator who is in charge of setting standards 
for the university-wide promotion and tenure process at your institution (often called 
the “vice provost” or “vice chancellor of faculty affairs”), or the individual(s) who set 
similar standards at the college or department-level. Share with them examples of 
world-class universities and scholarly societies (hint: there are a bunch in this chapter!) 
who find value in both public engagement and altmetrics to document the influence 
thereof. Offer to help serve on or consult for committees established to revise P&T 
preparation and evaluation guidelines in the future, with the aim of getting altmetrics 
documented and on researchers’ and reviewers’ radars. 

Alternatively, you can take a grassroots approach. Get on the agendas for campus- 
wide faculty union meetings and as many departmental meetings as possible, and 
talk frankly about the need to recognize the important work of public engagement. 
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Describe how altmetrics can help in evaluating one's engagement success in the P&T 
process, and offer to help incorporate them appropriately into P&T guidelines. 

Even if neither administrators nor faculty immediately warm to the idea of altmetrics, 
you've planted the seed. As altmetrics and their value in the P&T process become 
more widely known, those same skeptics may request your help in a year’s time! 

When advocating for the use of altmetrics in P&T, not only should metrics be 
mentioned. Remind reviewers and department heads of the importance of evaluating 
qualitative data-that’s where the interesting public engagement stories lie. Attention 
is not inherently positive and it is important to consider what is being said and not just 
how often research is being discussed. 

In the previously mentioned American Sociological Association’s report on public 
communication in promotion and tenure, the authors acknowledge the amount of 
time required for engaging in public outreach. Dr. Ernesto Priego, at City University 
London, also draws attention to the need to acknowledge and value the time that 
researchers are putting into public engagement. "Advocating for the consideration 
of altmetrics for promotion and tenure means we as a sector recognise that building 
engagement around scholarly outputs is time consuming and laborious." 

When advocating for the inclusion of altmetrics in promotion and tenure, you are 
giving your researchers the chance to be credited for the work they are likely already 
doing. As a librarian who’s now well-versed in the tools needed to use altmetrics 
appropriately in the promotion and tenure process, you can help further equip 
researchers to get that recognition. 

Chapter Summary 

• The use of altmetrics is a useful complementary source of information when 
helping researchers build Promotion & Tenure portfolios. 

• Any altmetrics should always be viewed and presented in context (e.g field, 
publication type, etc), as well as having been collected in a responsible way. 

• Tailor your advice! Consider the researcher, the department, and the P&T 
process, and adapt your process to meet their needs. 

• Altmetrics are especially useful to librarians who are likely to publish more “non- 
traditional” outputs. 

• Librarians can champion the use of altmetrics in the P&T process within their 
own institution. 

Further Reading 

1. Coates, H. (2015). “Advice from a Librarian: How to Do Successful Altmetrics 
Outreach." Altmetric Blog. 09 June 2015 

2. Priego, E. (2014). “On Metrics and Research Assessment." Ernesto Priego’s Blog. 
23 June 2014. 

3. Burnham, Joy J., PhD, Lisa M. Hooper, and Vivian H. Wright. (2012). “Top 10 

Strategies for Preparing the Annual Tenure and Promotion Dossier.” Faculty 
Focus Higher Ed Teaching Learning [website]. 25 Apr 2012. 
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Special Issues in Librarianship & Altmetrics 



Using Traditional Metrics 
and Altmetrics for Collection 
Development 

According to a recent survey, 74% of academic librarians in the US have regular 
collection development responsibilities. Altmetrics are just one of many types of data 
that they use to make decisions regarding purchasing and deselection. Each data 
type provides special insights into the use of materials within an institution. 

In this chapter, well discuss the relative merits of each approach, paying special 
attention to altmetrics and their applicability to the selection of journals, books, and 
emerging scholarly communication tools. 

Traditional approaches to using metrics for 
collection development 

The same survey referenced above found that usage statistics are the most often 
used metric consulted when making collection development decisions, followed by 
the journal impact factor and citation counts. 

Journal Impact Factor and related metrics 

What it is: The Journal Impact Factor is often used as a quick way to understand 
the influence of a journal, based upon the average number of citations any article it 
publishes can expect to receive. It’s a proprietary metric that calculates the average 
number of citations that research articles published in a particular journal have 
received in a two-year window. 

Limitations: The Journal Impact Factor can bea quickand easy way fora non-specialist 
to understand which are the most influential journals in a particular discipline, but it 

should not be used to understand the quality of a journal, contrary to popular belief. 
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Be sure to use the Impact Factor in combination with other metrics, to get a more 
complete understanding of the usefulness of the title. 

The Journal Impact Factor is more often used to evaluate journals in the sciences, and 
has been criticized for inferior coverage for non-English language journals (due to its 
reliance on Web of Science citation data). Because it is based upon citation counts, 
the Impact Factor shares many of citation counts’ limitations (see below). 

Where to access: Impact Factors for thousands of journals are available through a 
subscription to Thomson Reuters Journal Citation Reports. Free alternatives to the 
JCR include the Scimago Journal Rank and Google Scholar journal rankings. Both 
alternatives are based on citation data that often differs from Web of Science citation 
data, and thus can differ from the JIF. 

Citation counts 

What they are: Citation counts are article, book, or dataset-level metrics sourced 
from scholarly literature, whereby references to related studies are counted and 
aggregated. Generally, citation counts are interpreted to mean that a relatively highly- 
cited article is of high quality or influence. 

Limitations: Citations take months or years to accumulate, making them a poor 
choice for understanding the academic impact of recently published work. It is usually 
easier to find citations to journal articles (especially when using databases like Web 
of Science) than it is to find citations to books, datasets, and other “non-traditional” 
scholarly outputs. Citations can only highlight the scholarly influence of research; it is 
not possible to understand the “real world” influence of research using citations alone. 
For these reasons, it is generally recommended that citation counts be supplemented 
with other metrics when attempting to gauge the importance of research beyond 
academia, or the scholarly influence of recently published research. 

Research has shown that the motivations behind citations are many (negative citations 
account for 5-14% of the literature), so high citation counts should not always be taken 
at face value. 

Where to access: Citation counts are available for articles, books, and datasets 
through subscriptions to: 

• Web of Science 

• Scopus 

• Book Citation Index 

• Data Citation Index 

Citation counts are also freely available via Google Scholar (and the related desktop 
tool Publish or Perish). However, Google Scholar indexes content that is not always 
peer-reviewed, calling into question the quality or usefulness of the citations in 
understanding disciplinary impact. 
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Usage statistics 


Viewed O 


Total Article Views 


HTML Page Views 

PDF Downloads 

XML Downloads 

Totals 

284,081 

Aug 23, 2011 (publication date) 
through Aug 02. 2016 * 

PLOS 

250,148 

28,596 

485 

279,229 

PMC 

3,681 

1,171 

n.a. 

4,852 

Totals 

253,829 

29,767 

485 

284,081 


11.73 % of article views led to PDF downloads 


400k 



Months 


■ Compare average usage for articles published in 2011 in the subject area: 

[ Biology and life sciences ^ I Show reference set 


o 


Figure 13. Article-level usage statistics, reported by PLOS 


What they are: Usage statistics are metrics that are generally interpreted to measure 
readership of or interest in journals, articles, and other scholarly outputs. The most 
commonly reported usage statistics are page views and downloads, but other 
metrics (e.g., abstract views, search terms, location) are also collected and reported 
upon under the usage statistics label, depending upon the source. Usage statistics 
are either reported immediately or are updated at regular intervals (i.e. monthly), 
depending upon the source. 

Limitations: Usage statistics are especially susceptible to manipulation and gaming. 

This fact necessitated the creation of Project COUNTER, an initiative that sets 
standards for recording, cleaning, and reporting usage statistics, but that is not itself 
without limitations. Not all publishers are COUNTER-compliant. Most COUNTER- 
compliant reporting standards report on usage at the journal-level, rather than the 
article-level. 

Though they are commonly interpreted to directly measure readership, usage statistics 
actually only measure clicks and views and instead serve as an indicator for readership. 
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How often does your institution use the following methods 
for gathering electronic resource usage data? 

Manually collecting COUNTER _ 
data from vendor websites ~ 

Manually collecting non-COUNTER 
data from vendor websites ~ 

SUSHI via a commercial electronic _ 
resource management system 

SUSHI via a custom implementation - 


— 





■ Never 

1 1 Sometimes 

■ Often 

■_ 












i 






A custom usage counting system 
your library maintains on its own" 









0 20 40 60 80 100 120 

Number responding 


Figure 14. COUNTER data collection survey results from Welker, 2012 


Where to access: According to a 2012 survey, a majority of librarians collect usage 
statistics manually from vendor websites. Technologies like SUSHI, created to help 
automate the data collection process, are unfortunately rarely used. 

Reporting services do exist for COUNTER-compliant usage statistics: ProQuest’s 360 
COUNTER and similar commercial tools aggregate much of this data for its subscribers, 
and in the UK the Journal Usage Statistics Portal (JUSP) offers similar features. 
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Figure 15. Altmetric Attention Scores for multiple research objects displayed in Altmetric Explorer 
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What they are: Altmetrics are attention data from the social web that can help 
librarians understand which articles, journals, books, datasets, or other scholarly 
outputs are being discussed, shared, recommended, saved, or otherwise used online. 
They can be reported at the item-, journal-, or author-level. Altmetrics can shed light 
on non-traditional scholarly uses of research (e.g., when students have bookmarked 
an article that they find useful due to an excellent Methods section) but also on the use 
of research by the public, practitioners, and other stakeholder groups. These insights 
can be useful for librarians making collection management decisions for “applied” 
disciplines or professional degree-granting programs. 

Limitations: Altmetrics are a relatively young class of indicator, and as such not much 
is known about the meanings behind altmetrics. Therefore, quantitative altmetrics 
data (the metrics themselves) should only be interpreted to understand the volume 
of attention being paid to research. Manual work is needed to assess the underlying 
qualitative data that makes up the metrics (who is saying what about research). While 
altmetrics are good at identifying “trending” research, they have not yet been proven 
to be a good indicator for lasting, long-term impact. Librarians seeking to evaluate 
non-English-language materials will find that altmetrics coverage is currently limited 
for these outputs, no matter the altmetrics service used. 

Where to access: Currently, the only altmetrics aggregator that allows librarians to 
search by topic, journal, etc and also among an institution’s scholarship is Altmetric 
Explorer for Institutions. Limited analysis can also be done using the Altmetric Explorer 
for Librarians tool. Altmetrics for individual articles and books can be found using the 
Altmetric bookmarklet, and also (in the case of journal articles and select books) 
collected from many publishers’ websites. 

Example: Evaluating Journals 


Altmetric Explorer My workspaces Explore the data Media monitoring Explore my institution Help Loggedinasstacy0altmetric.com 


Articles mentioned at least once in the past Id 



Figure 16. Journal-level Altmetric Attention Scores displayed in Altmetric Explorer for Librarians 


Though altmetrics tend to be reported at the article-level (a phenomenon you’ve likely 
seen if you’ve ever encountered an Altmetric badge on a journal article’s webpage), it 
is possible to find journal-level altmetrics using the Altmetric Explorer database. 

At ER&L 2016, librarians from the Mt. Sinai Icahn School of Medicine explained how 
they use Altmetric Explorer data for collection development. 
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Journal-level usage statistics (usually provided by COUNTER-compliant publishers) 
and cost-per-use (a metric generated in-house, using Icahn School of Medicine 
purchasing data) are examined alongside aggregated, journal-level Altmetric scores 
(seen in the illustration above) to understand how different types of resources garner 
different types of attention, and thus may each have differing values to an organization 
like a medical school. 

Elsa Anderson (Deputy Director, Collections, Icahn School of Medicine at Mount 
Sinai) described the median Altmetric score as perhaps the most useful indicator for 
comparing journals with different levels of yearly publication volumes (because can 
you really compare raw, aggregated usage stats for a journal that only publishes 200 
articles a year to one that publishes 2,000 articles a year?) 

Moving forward, the team is very interested in combining disparate data types into a 
single, weighted indicator that allows them to accurately evaluate different types of 
publications (for example, the Pediatric Care Newsletter and the American Journal of 
Pediatric Care) at a glance. 

Example: Evaluating Books 

Let’s say there are a handful of ebooks in a package that you’re renegotiating with a 
vendor, and you’re not sure how useful they are to your community and beyond. 

Publisher-supplied ebook usage statistics are a solid real-time indicator for campus 
interest in these titles. These tell you whether or not the books are being accessed by 
your patrons (an indicator for recent readership). 

Beyond your campus, WorldCat-reported library holdings (i.e. whether or not these 
titles are owned by peer institutions can help you understand if these titles are popular 
elsewhere— perhaps an indicator for future interest in these titles. 

To find WorldCat library holdings, simply search for your title from the WorldCat 
homepage, then find it in the list of search results. On the item page, you can see the total 
number of libraries that own that book under the “Find a Copy in the Library” section. 

Alternatively, you might look up GoodReads ratings for the titles, or book reviews 
submitted to Amazon.com. 

On a side note, if you’re evaluating a book that’s been authored by someone from your 
own institution, PlumX Dashboards can aggregate the above book metrics for you. 

When combined with other data points (book reviews, faculty requests, etc), these 
metrics can make useful yardstick by which to make decisions about which books 
and packages to purchase. 

Beyond making purchasing decisions for books and journals, there are many other 
ways you can use altmetrics to inform your collection management decisions. 

On the following page, we’ve created an at-a-glance chart for the proper applications 
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of each metric type covered in this chapter. It offers general best practices for metric 
use, but keep in mind that there’s always exceptions to the rule. For example, citation 
counts may be useful for evaluating science articles in general, but not great at evaluating 
mathematics articles, due to disciplinary citation practices. You may wish to print the chart 
out and refer to it when making future data-backed decisions regarding your collections. 

Chapter summary 

• The Journal Impact Factor is a popular metric for making journal-related 
selection decisions, especially in the sciences. It should not be used to evaluate 
the value of individual articles. 

• Usage statistics (pageviews and downloads) are the most-used metric for 
making collections decisions, particularly for journals. They can be used to 
evaluate journals, articles, and many other types of research outputs. 

• Altmetrics are used by a small but growing number of librarians to make 
collection management decisions. They apply equally well to journals, articles, 
books, data, and other research outputs. 


Best metrics for collection development, at a glance 

The darker the shade of the intersecting cell, the more appropriate that metric type 
is for the specified use case. 



Journal Impact 

Citation 

Usage 

Altmetrics 


Factor 

counts 

statistics 



Usefulness, by discipline 


Sciences 
Social sciences 
Arts & Humanities 
Usefulness, by output type 


Books 
Journals 
Articles 
Data 

Software 

Recently published work 
Usefulness for understanding various influence types 


Long-term disciplinary influence 
“Real world” impact 
“Buzz” or popularity 


Legend 


Not at all useful 




Extremely useful 
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A Practical Guide to 
Altmetrics for Scholarly 
Communications Librarians 

You’re a scholarly communications librarian. Your job is to support researchers at your 
institution to successfully navigate the scholarly ecosystem of open access, research 
data management, digital publishing, research impact and metrics, copyright — and 
probably lots more. 

How do you make sure you are offering useful and tangible altmetrics support within 
your suite of scholarly communications services? 

This chapter focuses on examples and ideas for altmetrics services run by scholarly 
communications teams in academic libraries. We’ll cover the special role for Scholarly 
Communications Librarians in promoting altmetrics, specifically: embedding altmetrics 
services alongside open access support, integrating altmetrics into institutional 
repositories, and educating researchers on research data metrics. 

Why you’re in the perfect position to advocate 
for altmetrics 

As a Scholarly Communications Librarian, you’re probably already in an ideal position 
to promote altmetrics and share best practices across your institution. You’re 
already out there talking to researchers, advising on open access policies, providing 
institutional repository services and infrastructures. You’re in all the right meetings 
and have plenty of key contacts. 

The question is: how do you provide sound advice and promote altmetrics at the 
right point in the research lifecycle, as part of a robust, library-based scholarly 
communication service? 
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Let’s think about promotion. In her recent blog post on the Altmetric blog, Heather 
Coates (Digital Scholarship & Data Management Librarian at UPUI), talks about the 
importance of being a change agent: 

One advocacy tactic is to help researchers step back to see the value of all 
their scholarly products in new ways. One easy way to get them to think 
outside the box is to have them list all of the products resulting from one 
specific research project. This usually includes presentations, posters, white 
papers, policy reports. Sometimes faculty list code, models, data, and teaching 
materials, depending on their focus. With these specific products in mind, 
describe a couple of scenarios for how altmetrics provide data for individual 
items. This really helps them to understand the power of altmetrics. 

Is there anyone better suited to be a “change agent” than Scholarly Communication 
Librarians? I don’t think so! 

Start small 

You could begin by embedding altmetrics advice during your existing training 
sessions. If you’re running a session on registering for an ORCID identifier and Scopus 
author disambiguation, why not also provide an introduction to altmetrics and 
demo searching in Explorer for Institutions by ORCID identifier, or signing up for an 
Impactstory profile using ORCID? 

Connect alt to open 

Scholarly Communications Librarians are experts in broadening access to research, 
particularly through their support for open access publishing. Lucky for you, altmetrics 
can help you promote open access— they serve as a nice incentive for “opening up” 
research, as open access research has been shown to receive more attention online 
than toll-access articles. 

This blog post by Euan Adie compares the attention to open access vs. paywalled 
articles in a single journal. He found: “ Open access articles, at least those in Nature 
Communications, do seem to generate significantly more tweets - including tweets 
from people who tweet research semi-regular/y - and attract more Mendeiey readers 
than articles that are reader pays." Furthermore, a recent study found an open access 
advantage for a set of articles also published in Nature Communications in terms of 
citation, article usage and citations. 

Clearly, it pays for faculty to participate in so-called “Gold Open Access” publishing, 
where articles are made freely available immediately upon publication. 

How do you encourage researchers to “go green” by depositing open access versions 
of their research in your institutional research information management system or 
repository (aka Green Open Access)? 
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Altimetries in IRs 

One option is to embed Altmetric’s free badges for institutional repositories within 
academic institutions. Adding additional metrics to your institutional repository helps 
increase the visibility of altmetrics and enhance the data in your repository records, 

according to Lucy Ayre from the London School of Economics (who embedded the 
Altmetric badge alongside repository download stats). William Nixon at the University 
of Glasgow in his presentation, Altmetrics in practice: How are institutional repositories 
using altmetrics today?, describes the opportunities for adding value to existing 
services and empowering academic colleagues with Altmetric badge embeds. His 
presentation also includes examples of embedding Altmetric data alongside download 
statistics and funder metadata. 

Integrating Altmetric badges is relatively simple: all it requires is a single line of code or 
installing a plugin (depending upon the repository platform software you run). If you 
run an EPrints repository, take a look at this Bazaar Package for the Altmetric badges. 
Indiana University Bloomington added Altmetric badges to DSpace item records and 
even changed their item deposit workflow to allow researchers to opt-in to making 
badges visible; details for how they did that can be found on the IU Libraries wiki and 
in this presentation. BePress Digital Commons software offers the ability to integrate 
Altmetric badges for all customers. 

Another way to get altmetrics for your repository content is to install PlumX Metrics 
“Plum Prints”, which are available for PlumX subscribers. It includes similar functionality 
to the Altmetric badge, but highlights slightly different metrics than those Altmetric 
reports. Like Altmetric badges, they can also report metrics for both the publisher 
version of articles and preprint versions. “Plum Prints” can be added in any instance 
where the free Altmetric badges are suggested above— check out the Plum Analytics 
website for more information. 

Altmetrics in IRs aren’t just good for researchers, they’re also good for libraries for all 
of the reasons below: 

• Encouraging researchers to deposit open access versions of their research 

• More enhanced statistics help demonstrate the value and use of your IR content 

• Provide more detailed evidence for collection development 

Check out New Opportunities for Repositories in the Age of Altmetrics by Stacy 
Konkiel and Dave Scherer for further insight into leveraging altmetrics to leverage 
your IR. 
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Here are some examples of embedded badges on institutional repositories: 



Figure 17. Altmetric badges in University of Zurich’s IR 


( University of 

Reading 


Life | Study | Research | Business | About | A-Z 


CentAUR: Central Archive at the University of Reading 


^ UoR Home 


CentAUR Home 
Latest Additions 
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Browse by Year 
Browse by Division 
Browse by Author 



► Open Access 

► Research Data 

► Library 
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Intensification of winter transatlantic aviation turbulence in 
response to climate change 



Williams. P. D. and Joshi, M. M. (2013) Intensification of winter transatlantic aviation turbulence in response to climate 
change. Nature Climate Change, 3. pp. 644-648. ISSN 1 758-6798 

Full text not archived in this repository. 

To link to this article DOI: 1 0.1 038/ndimatel 866 

Abstract/Summary 

Atmospheric turbulence causes most weather-related aircraft incidentsl . Commercial aircraft encounter moderate-or- 
greater turbulence tens of thousands of times each year worldwide, injuring probably hundreds of passengers 
(occasionally fatally), costing airlines tens of millions of dollars and causing structural damage to planesl , 2, 3. Clear- 
air turbulence is especially difficult to avoid, because it cannot be seen by pilots or detected by satellites or on-board 
radar4, S. Clear-air turbulence is linked to atmospheric jet streams6, 7, which are projected to be strengthened by 
anthropogenic climate change8. However, the response of clear-air turbulence to projected climate change has not 
previously been studied. Here we show using climate model simulations that clear-air turbulence changes significantly 
within the transatlantic flight corridor when the concentration of carbon dioxide in the atmosphere is doubled. At 
cruise altitudes within 50-75’ N and 10-60’ Win winter, most clear-air turbulence measures show a 10-40% increase in 
the median strength of turbulence and a 40-1 70% increase in the frequency of occurrence of moderate-or-greater 
turbulence. Our results suggest that climate change will lead to bumpier transatlantic flights by the middle of this 
century. Journey times may lengthen and fuel consumption and emissions may increase. Aviation is partly responsible 
for changing the climate9, but our findings show for the first time how climate change could affect aviation. 

Item Type: Article 
Refereed: Yes 

Divisions: Faculty of Science > School of Mathematical and Physical Sciences > NCAS 

Faculty of Science > School of Mathematical and Physical Sciences > Department of 
Meteorology 


Figure 18. Altmetric badges in University of Reading’s IR 
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Embedding the badge helps leverage your repository and extend your existing service 
in useful ways for your researchers, and encourage open access repository deposit. 
Win-win. 

If your library hosts journals on behalf of your institution, you can embed altmetrics 
in Open Journals Systems either using the Altmetric badge embed code or the 

Plum Analytics OJS plugin-available from the University of Pittsburgh here. PKP 
are also developing an altmetrics plugin for OJS based on Lagotto, currently in beta 
development. 

Show me the data metrics! 

As a Scholarly Communication Librarian, you might also be running a research data 
management support service. This might include providing advice, training, technical 
infrastructure, data management plan guidance and policy support to researchers to 
help make their data well described, citable with persistent identifiers such as DOIs, 
preserved, reusable, reproducible and openly available wherever possible. 

Many institutions and funders now require research data assets to be made openly 
available according to policy requirements. A number of publishers also have data 
availability requirements: PLOS journals, for example, require authors to make all data 
underlying the findings described in their manuscript fully available without restriction, 
with rare exception. Helping academics to meet these requirements is a key element 
of an RDM service. 

How do you as a Scholarly Communications Librarian encourage participation in good 
data stewardship and help researchers get credit for the attention paid to their data? 

That’s where altmetrics come in. 

First of all-a brief summary of metrics for research data. Traditional metrics such 
as citations are available for research data and citation counts can be accessed via 
services such as the Data Citation Index. Citation metrics can help researchers get 
credit for academic engagement with their data. Altmetrics complement citations by 
helping researchers, institutions and funders track online activity, engagement and 
re-use of research data outputs, beyond traditional academic attention. 

The Making Data Count project investigated metrics for data, including surveying 247 
researchers and 73 data managers — finding that citations and downloads remain 
useful metrics to researchers and should be made available as broadly as possible. This 
is likely to develop in the future with the availability of a broader range of alternative 
metrics for data becoming more and more important to researchers and universities. 

As Stacy Konkiel discusses in her blog post on Tracking the impacts of data - beyond 
citations, there are three types of altmetrics for research data: “ repository usage, 
repository-sourced metrics (which often measure not only researchers' impacts, but 
also repositories’ and curators' impacts), and social web metrics (which more often 
measure other scholars' and the public's use and other interactions with data)." We’re 
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going to focus on the social web metrics-or altmetrics-and look at how you can 
embed data metrics in your RDM service. 

Examples of altmetrics for data might include: 

• Tweets sharing your open data sets (see example) 

• News coverage of your data set and findings (see example) 

• Facebook mentions of your data sets (see example) 

• Wikipedia references to your data (see example) 

• Blog coverage of your data sets (see example) 

Altmetrics services are great at aggregating this type of information for you, in one 
place. The Altmetric Details Page below for a Figshare data of 3D PDF dinosaur 
images includes Twitter, blogs, Facebook and news attention, including international 
coverage in the Los Angeles Times: 



Figure 19. An Altmetric details page for a popular, Figshare-hosted data set 


Your job as a Scholarly Communication Librarian is to educate researchers that 
altmetrics are available for data— most likely they do not know that they exist! You 
can promote altmetrics for data in a number of ways: 

If you have a data repository at your university, you could embed the Altmetric badges 
in each record to empower your researchers to track attention to their data. Take a 
look at this example on Cranfield University’s data repository and the University of 
Reading’s RDM FAQs on monitoring data mentions using altmetrics. 

You could also provide access to more powerful reporting services such as Explorer 
for Institutions, PlumX Dashboards or a researcher profile like Impactstory, which 

provides altmetrics for data sets and other types of research outputs. (Impactstory’s 
sister service, Depsy, also helps researchers get credit for software.) 
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Finally, if you’re running RDM-related presentations-maybe on the advantages of data 
sharing or introducing your new data repository-make sure you talk about altmetrics 
for data! 

Chapter summary 

• Scholarly Communications Librarians are in an ideal position to promote 
altmetrics and share best practices across an institution. 

• Start small by embedding altmetrics promotion during your existing training 
sessions. 

• Altmetrics can help you promote open access— they serve as a nice incentive 
for green OA IR deposit. 

• Embedding altmetrics in institutional repositories help enhance and strategically 
position your service. 

• Data metrics help you encourage participation in good research data 
management and help researchers get credit for the attention paid to their data 
outputs. 

Further reading 

• Kratz, J. & Strasser, C. (2015). “Making data count.” Sci. Data 2:150039 doi: 
10.1038/sdata.2015.39 http://www.nature.com/articles/sdata201539 

• Altmetrics in Practice: A Case Study: https://www.altmetric.com/blog/ 
altmetrics-in-practice-case-study/ 

• NISO Altmetrics Initiative: "ittp://www.niso.org/topics/tl/altmetrics_initiative/ 

• Piwowar HA, Vision TJ. (2013). “Data reuse and the open data citation 
advantage.” PeerJ 1:e175 doi: 10.7717/peerj.175 

• CODATA-ICSTI Task Group. (2013). “Out of Cite, Out of Mind: The current state 
of practice, policy, and technology for the citation of data” [report], doi:10.2481/ 

dsj.OSOM13-043 

• Costas, R., Meijer, I., Zahedi, Z., & Wouters, P. (2013). “The Value of research 
data: Metrics for datasets from a cultural and technical point of view” [report], 
Copenhagen, Denmark: Knowledge Exchange, www.knowledge-exchange.info/ 
datametrics 


Altmetrics for Librarians: A Practical Guide to Altmetrics for Scholarly Communications Librarians 


56 


Special Issues in Librarianship & Altmetrics 



Using Altmetrics 
in Digital Libraries 

Digital libraries are invaluable scholarly resources. Millions of dollars per year are 
poured into building digital library technology, digitizing content, and creating the 
open source infrastructure that supports countless researchers. But surprisingly, not 
much is known about how digital library collections are actually being used! 

In this chapter, we describe the current state-of-the-art for assessing the use and 
influence of digital library collections. Building upon current practices, we also 
recommend practical strategies for expanding digital library assessment practices 
using altmetrics and other data. By the end of this chapter, you’ll be well equipped 
to leverage usage statistics and altmetrics to measure the value of your digital 
collections, a natural first step to making it easier to win more resources, grants, and 
administrative support for your library. 


Measuring value in digital libraries: how it’s 
currently done 

As information experts, librarians are increasingly being named as on-campus 
resources for researchers and administrators interested in learning using research 
metrics. And yet we rarely use research metrics ourselves to understand the value 
of our collections and services, even for digital library content, whose value might be 
easily quantified. 

In this section, we outline how digital libraries currently measure value, libraries’ reasons for 
assessing their digital collections, and discuss various metrics’ strengths and limitations. 
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What do we measure? 

For digital special collections, we often track metrics for both digitized objects and 
descriptive information related to those objects, including: 

• Compressed and uncompressed versions of images, videos, and audio files; 

• Text files (available in XML, PDF, and HTML formats); 

• Descriptive information about the object captured in a variety of metadata 
standards; and 

• Contextual information about an object or a collection described in 
accompanying essays, timelines, and visualizations. 

It would come as no surprise to anyone who has worked with digital collections to 
say that digital library content is heterogeneous and, in many cases, complicated to 
measure. 

Why do we measure? 

Demonstrate value 

Often, we measure our collections to prove their value to stakeholders, including donors, 
faculty, and library and university administration. With the tightening of belts across 
academic divisions, libraries also use metrics to document the "ROI” of staff hours and 
collections dollars (Mays, Tenopir, & Kaufman, 2010; Dinkins & Kirkland, 2006). 

At an institutional level, attention metrics (tweets, Facebook shares, mainstream 
media mentions, etc.) can be appreciated in terms of public relations, especially for 
land grant institutions with a community engagement mandate. It may be difficult for 
library administrators to see the value in attention metrics beyond marketing, but one 
should keep in mind that scholarly exchanges are increasingly taking place via online 
networks, including on social media platforms (Lavoie et at, 2014). As such, attention 
metrics may evidence scholarly engagement with digital library collections. 

Digital special collections can have value to the public, beyond their use for research 
and scholarship. Many collections are reused by the casual reader in ways that can 
leave traces on the Web, like unexpected references to collections in the form of 
memes, fan websites, and other pop culture formats. 

Make decisions 

Metrics are used every day by libraries to determine how collection development 
budgets are administered, whether new staff are hired to support particular areas 
of study, and which monographs are worth keeping versus deaccessioning. Our 
dependence on metrics to make important decisions naturally extends to digital 
library administration. 

Metrics are often used to monitor user experience and improve the functionality 
of library web sites for digital library content (Dalmau & Hardesty, 2008). Metrics 
can also help plan for future digitization projects (Digital Library Federation Forum 
Working Group Session Notes, 2013). 
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Who are we measuring value for? 

Tracking, collecting, and analyzing metrics become more meaningful once we 
understand the stakeholders that may use these metrics. 

Below, we provide examples of various stakeholder groups and the metrics they 
might use. Note that your institution’s stakeholders may differ based on your library’s 
mission, mandates, and the content that you publish. 

Librarians & Archivists 

Librarians and archivists might use metrics to help make decisions on which digital 
library content to deaccession or invest in. Metrics can guide library outreach and 
promotion efforts, helping to market digital library content to the public, scholars, and 
even donors. Metrics can also be useful when building a case to library or university 
administration for increased resources. Patterns of use may indicate ways to capitalize 
on public engagement for projects like crowdsourcing metadata or transcription. 

Library Administrators 

Similarly, administrators may use metrics to make decisions on how to allocate 
resources, particularly to justify costs of building and sustaining a digital library. 
Metrics can also be a handy way to demonstrate value to funders, whose grants often 
enable libraries to digitize collections and build critical digital library infrastructure. 

Funders 

Funders may use metrics to determine the value of digitization projects they fund. 

Funders from other sectors are already using research metrics in this manner. 

Scholars 

Scholars may use metrics to evaluate digital library content, when deciding whether 
or not to download, cite an item, or otherwise reuse it. 

Educators 

Educators might use metrics to teach information literacy concepts, when teaching 
students how to identify and evaluate potentially high-quality digital library content. 

Students 

Library and information science students who have contributed to digital collections 
as interns or hourly workers could benefit from metrics that can demonstrate the 
value of the metadata, markup, and scanning they have done. 

How are we measuring value? 

Digital libraries currently tend to use a mix of quantitative and qualitative data to 
understand the influence of their collections. There are no standard practices, making 
assessment strategies at many institutions locally relevant to campus priorities and 
often ad hoc in nature. 
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Quantitative metrics 

Usage statistics and content-related metrics were historically the core metrics tracked 
by digital libraries. It’s worth noting that at many institutions, such tracking is not 
systematic nor transparent. 

Usage statistics 



Figure 20. Usage statistics displayed on a University of Southampton repository item record 

Usage statistics include, but are not limited to, page views, downloads, search queries 
and other data related to users’ access of digital library content. Usage statistics were 
once the sole domain of system administrators, pulled from server logs and often 
only accessible to librarians from the command line. However, more digital collections 
are beginning to publicly report download and pageview statistics by default (an 
example can be seen in Figure 20). 

Usage statistics are commonly sourced from web logs and Google Analytics data. 
We often use these systems to patch together unique reports on our collections and 
cyberinfrastructure. Some examples of useful metrics that can be sourced from web 
logs and Google Analytics include: 

• Operating systems and devices used to access content; 

• Time-based metrics (i.e., times of day that digital libraries see the most traffic; 
used internally to optimize server maintenance times); 

• Landing pages for most popular content; and 

• Collection-level usage statistics. 

Content-related statistics 

Content-related metrics report on the amount of files in a digital library and their 
characteristics (file size, format, color profile, etc.). 

Digital special collections content statistics are often focused on the “repository” 
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or “workflows,” including the total number of objects in a collection, file types, 
characteristics of metadata, benchmarking, and so on. 

It’s unclear whether or not these metrics are useful for assessing the influence of 
content; instead, these metrics are mostly useful for preservation, maintenance, and 
collection development tasks. 

Qualitative data 

Most qualitative data collected on the use of digital library content are based on 
interviews and ethnographic usability studies about patrons’ use of these systems 
(Jeng, 2005; Cunningham, Reeves & Britland, 2003). 

Quotes from users and general themes to user feedback are also often collected in an 
ad hoc manner, usually when users contact collection administrators to request reuse 
permission for digital content. 

Whether or not content appears in syllabi (and if those syllabi are from courses offered 
at the same institution as the digital library that hosts the content) is also sometimes 
used to demonstrate the educational value of digital library content. 

Limitations of our current metrics 

Lack of context 

Currently, digital library-collected metrics are not able to capture the context 
surrounding consumption and reuse. For the most part, we do not know who is using 
our content, nor for what purposes— though at least one study has shown that it is 
possible to create narratives based on server log and usage data (Angelo, 2014). 

This lack of context means that librarians cannot easily gauge the varied types of 
significance that collections may have. “Hits” alone do not always convey value; we 
need to enhance our current use of metrics— and perhaps find new metrics— that can 
help us better pinpoint the value of digital library content. One way to do so would 
be to find metrics or qualitative data that help us understand the unique ways that 
scholars and the public use and benefit from our content, similar to the manner that 
researchers have been able to find “flavors of impact” for STM literature by measuring 
the correlations of particular types of metrics (Priem, Piwowar & Hemminger, 2012). 

Gaming 

How can digital libraries record meaningful statistics if we are unable to better detect 
and prevent the “gaming” of usage statistics? 

Gaming is the artificial inflation of metrics. It can be achieved by both manual (repeated 
human-initiated requests) and automated means (repeated computer-initiated 
requests, often achieved with the help of “bots”). Intentional gaming is currently less 
of concern to digital libraries, which tend to worry more about the effective filtering 
of robots that are deployed to index digital library content (more on this below). 
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Ways to correct “gamed” metrics include: 

• Banning the IPs of potential gamers and bots, 

• Using altmetrics reporting services like Altmetric that account for self-tweeting 
and other types of self-promotion (Adie, 2013), and 

• Making available the underlying, qualitative data, so users can see exactly who is 
tweeting, blogging, citing, and bookmarking digital library content. 

Filtering out the noise 

Digital libraries face the challenge of filtering out hits and queries for content initiated 
by “spiders” and institutional testers. Crawlers can be identified, but mass downloaders 
are harder to identify, as they change IPs frequently. Browser caching and dynamically 
assigned IPs can also make counts inaccurate and make it harder to profile and track 
our users. Multiple http requests for a single page (where separate “hits” occur for 
web page graphics, css, external javascript files, etc) sometimes can inflate metrics 
for digital collections. 

Associating multiple versions of an item & means of access 

An additional challenge to creating reliable metrics for digital libraries is that multiple 
versions of content often exist. For example, for a single image within a digital 
collection, there are often separate files in the form of an image’s thumbnails, low- 
resolution copies, high-resolution copies, and preservation-ready “master” files. How 
can you accurately measure the value for a single item, given all the versions of that 
item that exist and have associated metrics? 

Institutional culture & other influencers of attention 

Marketing and outreach have an important effect on the attention that digital library 
collections receive, yet each institution’s ability to perform marketing, outreach, and 
SEO optimization on collections differs. This makes cross-institutional comparisons 
difficult. 

Institutional cultures of assessment are important to consider. Many institutions do 
not have assessment practices beyond ad hoc ones, or have not adopted existing 
best practices. Creating longitudinal sets of metrics for a long-term view into the 
value of a collection is crucial, but not practiced consistently. 

Recommendations 

In this section, we propose important metrics to capture, methods for collecting these 
metrics, and guidelines for how to apply metrics when reporting value. In addition, we 
provide some related resources that can provide further guidance on the integration 
of such metrics into digital library assessment and reporting practices. 

What should we measure? 

In addition to the commonly used metrics identified above— which can be very valuable 
for digital library assessment— libraries should consider tracking the most famous 
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traditional scholarly metric: the citation. Citations can uncover scholarly attention, and 
are an excellent signal for the long-term disciplinary value of digital library materials. 

Altmetrics are data related to sharing and reuse of scholarly content online. They 
can shed light on diverse audiences for digital library content (the public, policy 
makers, educators, etc.). Because they are reported directly from the websites where 
discussions of research are happening, they can accumulate quickly. They are often 
combined with citations and usage statistics when reported out through third-party 
services like Altmetric.com and PlumX. 

Beyond raw counts, contextual information is often required to understand the value of 
scholarship. (Who is interested and why? Are they a scholar or a member of the public? 
Are they using a University Archives photograph or pre-print of a physics paper?) 
Usually, we must manually review contextual information to understand context. 

Context is also important in terms of longitudinal use. While some collections may 
see a short spike in popularity, others garner incremental interest over the years, and 
could be said to have greater value in the long term. 

Context should be easy for both librarians and end-users to understand, especially 
if collection and item metrics are made publicly available. Ways of adding context 
include listing clearly displayed: 

• Percentiles (“This image has 291 views. That’s more than 97% of other images 
in this collection, and 76% of all images held by the University Libraries’ Digital 
Collections”), 

• Demographic information (“Of the 9,815 views this video has received, 25% are 
from users in the US, 17% are from users in the UK, and 58% are from elsewhere 
in the world.”), and 

• Data collection parameters (“Download and pageview metrics have been 
collected since the item’s deposit on 17 June 2012. Twitter mentions have 
been collected since September 30, 2013. Blogpost mentions have been 
collected since September 30, 2012, and include only posts that occur on the 
ResearchBlogging network.”) 

To summarize: where possible, libraries should display digital library citation metrics 
and altmetrics alongside important contextual information. 

Types of Assessment Data & Their Uses 

Below we present various types of metrics organized by their quantitative and 
qualitative characteristics. We do not intend to dichotomize these characteristics, but 
instead encourage a combination of both approaches. Caveats for capturing these 
metrics exist and are highlighted below. 

The list below is not comprehensive, but instead reflects metrics that are commonly 
used and easily tracked, and which can be used to understand attention paid to digital 
library content by various audiences. 
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Quantitative metrics 

Quantitative metrics are arguably easier to capture than other types of data. Though 
they may not be fully indicative of how digital library items are used, they are indicative 
of overall volume of interest in digital library content. Furthermore, by combining 
traditional web analytics with altmetrics, a fuller picture of end-user consumption can 
be understood. 

Page views are commonly tracked in web analytics software, and are often called 
“impressions”. Transaction logs and Google Analytics are excellent means to track this 
metric and the related session length metric. Note that the duration of time spent on 
a given page may reveal clues about depth of engagement with content, but should 
be interpreted with a grain of salt, as this information could be skewed by multitasking, 
tabbed browsing, and bandwidth issues. 

Visits, especially returning visits, are session-based metrics that track a user by IP and/ 
or persistent cookies, and can provide some indication of engagement. Dynamic IPs, 
different people using the same computer, or one person using multiple browsers can 
impact data collection. Privacy implications of the use of cookies are a possible issue 
here. Transaction logs and Google Analytics are excellent means to track this metric. 

Referring sites straddle the quantitative and qualitative realms in that a subset of 
referring sites serve as some indication of scholarly (i.e., Wikipedia and Google Scholar) 
or popular relevance (i.e., news outlets and Reddit) that could be further traced for 
citations and references in context. Google Analytics reports referring sites well. 

Downloads are another common metric for tracking use of digital library content 
that convey, at the very least, that items are being successfully exposed across 
the broader Web. In some ways they are akin to circulation counts, which have 
historically been regarded as an important metric of use for libraries. Accurately 
tracking downloads can be problematic for digital special collections that are 
stored in digital object repositories and are referenced via persistent URLs (PURLs). 

Server transaction logs and Google Analytics can track downloads. A common 
misconception is that Google Analytics cannot be used to track download statistics 
for content; some have configured the platform to do just that (Knowles, 2012). 

The COUNTER guide to implementing auditable download metrics is useful for 
more information. 

Direct links are important to track because a significant amount of link sharing 
happens in emails. It is also possible that links might be typed directly into the browser. 
(Admittedly, in the age of Google, that behavior is less likely to occur than in previous 
years.) To a limited extent, direct links may be trackable in Google Analytics. 

Shares on social media provide a powerful mechanism for circulating content across a 
vast network of people. The act of sharing can sometimes signal implicit endorsement 
of the content being shared. Altmetrics aggregators are well-equipped to track shares 
on social media. 
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Saves, Favorites, and Bookmarks can capture interest in a given item, and in some 
cases the intent to use items in future teaching, Altmetrics aggregators are well- 
equipped to track saves, favorites, and bookmarks on social media and in scholarly 
social bookmarking sites. 

Adaptations relate to the creation of derivative works: new research or creative 
works based on existing digital library images, data, research, software, etc. Tracking 
adaptation is difficult to do, even in systems that provide mechanisms for doing 
so (i.e. forking in GitHub). That’s because while some adaptations may be created 
in public (for example, a forked, public software repository might be adapted into 
another public software repository on GitHub), others might download digital library 
content in order to adapt it privately (on their desktop, or in Dropbox, etc.). For digital 
library content that requires end users to manually offer attribution (i.e. citing a 
digital collection in a book chapter), the problem can be more pernicious. Altmetrics 
aggregators can track some adaptations, as can thoughtfully constructed Google 
Alert/Mention search queries. 

Requests for hi-resolution digital library content, submitted via automated means, 
could bean indicator of later citations or reuse and adaptations. Further study is needed. 
In digital libraries where the request process is automated (for example, requests are 
sent in via a “contact” form on a website), Google Analytics and transaction logs may 
report this information. 

However, these requests are usually collected on an ad hoc basis, as they’re often 
submitted via email. Using customer support email systems like Zendesk, Freshdesk, 
or other ticketing systems to manage these request emails may help librarians more 
easily track the overall volume of requests over time. 

Citations help us understand the use of our digital libraries in a scholarly context, 
particularly when cited in books and journal articles. Citations to digital library content 
can be difficult to uncover, however, as it’s unclear to what extent citation indices like 
Scopus and Web of Science can create accurate citation reports for digital content 
that isn’t a book, book chapter, or journal article. 

Visitor demographic information is another metric of interest to libraries. Demographic 
information like age and user interests can be sourced from third-party services like 
Facebook or Google (which are sometimes used to allow visitors to login to library 
websites), from IP addresses that help determine users’ location, or even from library- 
administered surveys. There are obvious privacy implications to tracking visitors’ 
demographic information. 

Qualitative data 

The use of qualitative data for assessment sometimes require manual collection and 
review, or personal engagement with a digital library’s users. 

Mentions can be as informal as a “shout-out” or as formal as a citation, though in 
either case the mention may not be constructed in easily traceable ways (i.e., citing 
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a canonical URL or persistent identifier). In venues like Twitter and Wikipedia, where 
mentions are more easily tracked and aggregated, this data can be easily harvested 
to better understand context: what is being said about a particular item? And who is 
involved in the discussion? 

Mentions have a huge potential for understanding value, but are harder to trace as 
they can be appear on the Web in many forms: course syllabi, blog posts, policy 
documents, and news articles (just to name a few). Google Alerts and fee-based 
services like Mention provide ways for gathering information about an item, project 
or person of interest, using search terms and keywords. Altmetric gathers mentions 
that occur in policy documents and mainstream media, particularly for items with 
persistent identifiers. 

Reviews or comments provide another avenue for determining value. The volume of 
comments often does not matter as much as the nature of the comments. In addition, 
a commenter’s identity can sometimes be equally important when analyzing comment 
content. To a limited extent, altmetrics aggregators can collect this information in the 
form of blog posts, blog comments and Twitter mentions. 

Reference inquiries regarding digital library content should be factored into 
assessment; they often provide a story of scholarly use and engagement beyond web 
analytics. They also create opportunities to follow-up with users to learn more about 
their research interests with respect to the digital library resources. Reference inquiries 
can be used to justify the allocation of staff time at the launch of collections, when 
questions from the public are likely to circulate. These are often collected and recorded 
on an ad hoc basis, being as they’re often submitted via email, telephone, or in person. 
Customer support services like Zendesk are also appropriate for consideration in 
managing these inquiries, to make it easier to track their volume over time. 

Case study: The Biodiversity Heritage Library 

About the Biodiversity Heritage Library 

The Biodiversity Heritage Library (BHL) is a consortium of worldwide libraries, 
headquartered at the Smithsonian Libraries in Washington D.C. Launched in 2007, 
the BHL online resource site receives over 110,000 unique visitors a month, and 
currently includes over 49.7 million pages of natural history literature from from 
over 109,000 titles. 

The library’s mission is to inspire discovery through free access to biodiversity 
knowledge; anyone in the world, wherever they are, should be able to make use of 
their content. Access to such information is a particular necessity to biodiversity- 
related sciences, as historical data and species classification underpins the work 
that scientists are doing today. 

Why the interest in altmetrics? 

In 2014 the BHL received an Institute of Museum and Library Services grant to 
transform BHL into a more social, semantic digital library. As part of this “Mining 
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Biodiversity” project, they are further integrating the BHL with social media, 
enhancing site search functionality, and improving their semantic metadata. 

“We wanted to see where people were talking online about our content - and help 
our readers see those conversations too.” 

Crucially, Grace Costantino (Outreach and Communication Manager at BHL) and 
her colleagues are keen to find ways to make it easier for their audience to discover 
more info about their collections, and share thoughts and knowledge about those 
collections using social media. The first step they took to encourage this was to 
add better sharing buttons to BHL, but they also wanted to more easily capture 
the online conversations surrounding their content and let others see what people 
had been saying. 


Implementation 

Although some of the BHL content is assigned a DOI, this practice is not consistent 
across all of their content. Grace and her team worked with Altmetric to instead 
track mentions to their content based on the URI (the unique identifier) of each 
piece of content. 


BHL About Help Feedback I3DQO 



Figure 21. How the Biodiversity Heritage Library have integrated Altmetric badges into their 
digital library 


BHL has now launched the Altmetric badges across their online platform, and 
are using the Altmetric Explorer internally to monitor and report on the online 
attention across their collection. 


They also added an overview of the Altmetric data and what it offers their readers 
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to their Wit , helping users of the platform understand what the data shows and 
how it can be interpreted. 

Up and running 

In July 2015, Grace reported that the BHL are finding the Altmetric data really 
valuable for discovering conversations that they didn’t know were happening - 
particularly as lots of people will share a link to the content they are talking about 
but don’t necessarily mention the BHL, making this activity difficult to track by 
keyword alone. 

To announce the roll-out of the Altmetric badges, Grace and her team put together 
a program of blog and social media content, and also included announcements 
in their newsletters and quarterly reports. Already they are using the Altmetric 
Explorer to identify what types of books are popular with audiences, and making 
additional efforts to offer similar content. 

“Through the Altmetric data we identified that our marine books in particular are 
really popular. Information like this helps us better tailor our posts and the content 
we share to ensure maximum engagement." 

With altmetrics providing an up-to-date measure of the success of their ongoing 
promotional and engagement efforts, in the future the BHL hopes to use altmetrics 
to help determine ongoing outreach and engagement strategy to ensure the 
continued awareness and success of their valuable content. 

The content of this case study has been adapted from a July 2015 Altmetric blog 
post, "Biodiversity Heritage Library highlight online attention surrounding their 
books with Altmetric badges" 


How to collect assessment data 

There are two important considerations for devising an assessment strategy: how 

often to collect your data and which tools to use to collect it. 

Timing 

For institutional use: It can be useful to collect and analyze metrics at times that 
coincide with both annual, library-wide internal reviews and external reporting events 
(like for ARL reports, LibQual statistics, and so on). That way, you can reuse the 
metrics collected for both purposes. 

For end users: Delivery of content for user-facing stats should be immediate. For 
example, “This item has been downloaded x times” is a metric that’s more useful 
when reported in real time. If manual intervention is required to prepare metrics (such 
as to parse server logs for relevant information), metrics should be regularly delivered 
(i.e. weekly or monthly) and transparently reported, so users can understand what 
they are looking at and can evaluate the usefulness of those metrics accordingly. 
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Tools 

Following are recommended tools for getting started with data collection. A holistic 

evaluation framework that librarians might also find useful is JISC’s Toolkit for the 

Impact of Digitised Scholarly Resources (TIDSR). 

Web server logs 

• Metrics reported: downloads, referring sites, visits, pageviews, limited 
demographic information. 

• Be sure to export information at regular intervals, as consistent collection is 
important for longitudinal analysis. 

• Web server log data often adhere to certain formats (Apache Custom Log 
Format, W3C Extended File Log Format, etc.) and can be processed and 
visualized for human consumption with the help of tools like Webalizer, 

AWStats, and Tableau. 

• Tableau is especially useful for web server log analysis, grouping, and 
visualization by creating dashboards, user population assessment, and usage 
over time. 

Google Analytics 

• Metrics reported: downloads, referring sites, visits, pageviews, limited 
demographic information. 

• Google Analytics has some dashboard functionality that’s useful for saving 
elements you want to review regularly. GA is also useful for longitudinal analysis, 
showing big trends in traffic and use. 

• For digital collections: Szajewski (2013) has written an excellent guide to using 
the tool to measure the value of digital special collections. 

Citation databases 

• Metrics reported: peer-reviewed journals and books that cite digital library 
resources. 

• Citations can be be sourced from subscription databases like Scopus and Web 
of Science, or from free platforms like PubMed, Google Scholar, Google Books, 

and CrossRef. Often, specialized searches are required to find citations to digital 
library content: “cited reference search” can be used in Web of Science, and 
free-text search for digital library or collections’ names can be employed in 
other databases to find references to digital library content. 

• Citations are much easier to track when persistent identifiers like DOIs are used 
by whomever is citing digital library content. 

Altmetrics aggregators: Altmetric & PlumX 

• Metrics reported: Shares, saves/favorites, adaptations, mentions, and some 
other quantitative and qualitative data sourced from the social web. For full lists 
of each service’s metrics, visit their respective websites. 

• These services collect data from across the web related to any scholarly 
outputs, which can be displayed in embedded badges (Altmetric) or widgets 

(PlumX). 
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• Altmetric collects altmetrics for any content in a subscriber’s digital special 
collection. Plum Analytics has not yet confirmed whether PlumX is equipped to 
track digital special collections. 

• Both aggregators provide important qualitative data behind the numbers they 
report, although to varying degrees. For example, in both altmetrics tools, in 
addition to seeing that items in your digital library have been mentioned 17 
times on Wikipedia, you can also see exactly what has been written about them. 
However, PlumX does report some metrics that do not have the underlying 
qualitative, auditable data available for review. 

Altmetrics data via social media APIs 

• It is also technically possible for digital libraries to connect with individual social 
media platforms’ APIs to search for mentions of their content. This might be 

a good option for digital libraries that do not issue permanent identifiers like 
Handles or DOIs; they could monitor social media sites for mentions of relevant 
URLs, instead. 

• The main drawback to this option is the the developer time required to build 
customized solutions for each digital library. It could possibly result in much 
duplicated effort. 

• Another possible drawback are the limitations placed on search APIs by 
platforms themselves; for example, Twitter’s search API is typically restricted to 
fetching data from only the previous week, and the API’s rate limits restrict the 
retrieval of large amounts of data at once (Kinsley, 2014). 

Qualitative data via Google Alerts and Mention 

• Track when your content has been shared on the web by setting a Google Alert 
or Mentior alert for your: 

• digital library’s name 

• digital library’s base URL 

(http://collection1.libraries.psu.edu/cdm/singleitem/collection/amc/id/314/ 

rec/1), 

• your repository’s Handle or DOI shoulder 

(https://scholarworks.iu.edu/dspace/handle/2022/9564 and 
http://hdl.handle.net/2022/9564; http://dx.doi.org/10.5061/dryad.pp67h 
and http://datadryad.Org/resource/doi:10.5061/dryad.pp67h), or 

• special URLs created for collections 

(http://webapp1.dlib.indiana.edu/vwwp/welcome.do) 

• For important collections, you might also want to set alerts for titles or names 
relevant to those collections (i.e. for Penn State’s “Advertising Trade Cards from 
the Alice Marshall Women’s History Collection,” they might also set alerts for 
“Advertising Trade Cards” and “Alice Marshall”). 

• Google Alerts is free to use; Mention is a subscription service. 
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Chapter Summary 

• Most digital libraries currently use relatively basic, minimally data-backed 
assessment strategies. 

• Adding altmetrics, citation data, contextual information, and other data to 
assessment practices could vastly increase libraries’ understanding of how their 
digital collections are being used and by whom. 

• A number of free and subscription tools can make it easy to automate the 
collection of data. Analyzing the resulting data at regular intervals can keep 
assessment projects manageable. 
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The Future of Altmetrics 



A Short Guide to Keeping 
Up-To-Date on Altmetrics 

The field of altmetrics is fast-changing, with research coming out weekly that gives us 
new insights as to altmetrics’ many meanings, interpretations, and uses. 

Plus, altmetrics themselves are ever-changing: new data sources are constantly 
appearing and often being deprecated. 

Keeping up with all these changes can be a bit of a bear. We’ve compiled a list of 
resources (social media streams, conferences, journals, and researchers) that can help! 

These resources are current as of August 2016. 

Journals to read 

High-quality altmetrics research is often published in the following journals: 

• JASIS&T 

• Scientometrics 

• Journal of Informetrics 

• PLOS ONE (specifically, the “altmetrics”, “bibliometrics”, and “social media” 

subject areas) 

Frontiers in Research Metrics & Analytics is a recently launched journal that will likely 
include relevant altmetrics research, as well. Open Access altmetrics research is 
also often shared in the Digital Libraries section of Arxiv and on many researchers’ 
personal websites. 
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Researchers to follow 

While many researchers dabble in altmetrics research, some scientometricians 
regularly publish valuable insights in the field. Here are a few to follow: 

• Lutz Bornmann 

• Timothy D. Bowman 

• Rodrigo Costas 

• Stefanie Haustein 

• Kim Holmberg 

• Isabella Peters 

• Cassidy Sugimoto 

• Mike Thelwall 

• Zoreh Zahedi 

The researchers listed above are by no means the only scholars contributing to 
altmetrics. We encourage you to explore the field further using article recommendation 
services like Sparrho and Google Scholar (and of course via good-old-fashioned 
expert search skills, which are in no short supply for librarians!) 

Social media spaces to monitor 

Twitter is the single best place to stay up-to-date with altmetrics research and industry 
news. Follow the #altmetrics and #altmetria hashtags, the Altmetric Ambassadors and 
Bibliometrics Experts lists, and the below Twitter accounts for the latest information: 

• Ronaldo F. Araujo 

• Isidro F. Aguillo 

• Keita Bando 

• Jason Priem 

• Cassidy Sugimoto 

• Organizational accounts from Altmetric, Impactstory, and Plum Analytics 
Relevant research occasionally surfaces on Academia. edu’s “Altmetrics” section, as well. 
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Conferences to attend 

There are a growing number of conferences devoted solely to altmetrics and the 
related fields of bibliometrics and webometrics: 

• Altmetrics Conference (also known as the X:AM conference series): This 
meeting usually occurs in Europe in the autumn and includes a nice mix of 
librarians, researchers, and scientometricians. 

• AltmetricsXX workshops: This one-day workshop is now paired with the 
Altmetrics Conference and highlights cutting-edge research in altmetrics. It is 
more research-heavy than the X:AM meeting. 

• SIG METRICS workshop and ASIS&T Annual Conference: The ASIS&T 
SIG METRICS interest group usually organizes a one-day workshop at the 
ASIS&T Annual Meeting, where scientometrics research is shared in general. 
Altmetrics research is often presented here, and to a lesser extent, it is also 
included in the program of the larger ASIS&T Annual Conference. 

• Science and Technology Indicators Conference and the International Society 
for Scientometrics and Informetrics (ISSI) Conference: These conferences are 
held throughout the world and often include altmetrics research in addition to 
bibliometrics and webometrics research. 

If you are unable to attend these meetings in person, don’t worry: proceedings, 
recordings, and conference tweets can often be followed from afar. Check out each 
meeting’s website for more information. 

The attitude to maintain 

If there’s one thing that can help you stay up-to-date on altmetrics, it’s understanding 
that what we know about this data is likely to change a lot over time! 

You should always continue to challenge your own opinion of altmetrics, and regularly 
ask yourself: How has the field changed in the last few months? What research has 
emerged on the types of impact and altmetrics that are relevant to me? What’s 
changed in the altmetrics industry that’s worth knowing about? 

Chapter summary 

• The field of altmetrics is relatively young and fast-changing. A number of 
resources can keep you up-to-date with minimum effort. 

• Reading specific journals and following a handful of recommended researchers 
is the surest way to find the latest, best altmetrics scholarship. 

• Twitter is a great social media venue for discovering altmetrics news, and lots of 
research is also shared on Academia.edu. 

• An increasing number of altmetrics conferences worldwide can be attended or 
followed from afar. 


Altmetrics for Librarians: A Short Guide to Keeping Up-To-Date on Altmetrics 


76 


The Future of Altmetrics 



Where the Field May Be Headed 

Though it’s hard to predict where altmetrics in libraries may be headed in the next few 
years, it’s possible that the field in general may soon develop in the following ways. 


Moving beyond metrics 

Many in the altmetrics community have acknowledged that numbers— while useful for 
helping to discover pockets of attention— usually aren’t suitable as impact evidence, 
by themselves. When making an argument for the influence of one’s research, the 
amount of discussion about research (as documented using metrics) tends to be less 
useful than the substance of that discussion (who is talking and what they’re saying). 

In the near future, we hope to see researchers, administrators, and librarians looking 
beyond raw metrics to find evidence of meaningful engagement when evaluating 
research using altmetrics data. Full-text altmetrics data will be used to help scholars 
craft their own narratives of influence, rather than just a list of numbers to display 
alongside the Journal Impact Factor on one’s C V. 


More context 

In the limited cases where metrics themselves may be used as impact evidence, 
context will be key. This context will likely be provided in at least two ways: through 
the greater use of percentiles to show the relative performance of research, and 
through semantic enrichment. 

Experts generally recommend the use of percentiles to give context to metrics. So, 
rather than saying, “This book was mentioned in 5 news articles,” one should say, 
“This book was mentioned in 5 news articles, placing it in the top 5% of all articles 
published in The Journal of Lilliputian Economics in 2014.” Percentiles generally make 
comparisons between research published in the same journal, discipline, or year, and 
it’s also possible to make comparisons at the author-level, using authors’ discipline, 
nationality, gender, or other variables. 


Altmetrics for Librarians: Where the Field May Be Headed 


77 


The Future of Altmetrics 


Two of the leading altmetrics services, Altmetricand Impactstory, each offer percentiles 
for their data. Altmetric’s percentiles use the Altmetric Attention Score (a weighted 
indicator for the volume and nature of attention that a research output has received) 
to make comparisons along four bases: comparisons to all 5+ million research outputs 
we've recorded as having online attention; comparisons to other outputs published in 
the same journal; comparisons to other outputs of the same age; and comparison to 
other outputs of the same age, published in the same journal. 


TWITTER DEMOGRAPHICS 


MENDELEY READERS 


ATTENTION SCORE IN CONTEXT 

V 


This research output has an Altmetric Attention Score of 2656. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score was calculated when the research output was 
last mentioned on 23 June 2016. 


#57 

of 5,41 4,487 outputs 


#1 

of 1,310 outputs 


#5 

of 133,145 outputs 


OUTPUTS OF SIMILAR AGE FROM BIOCHEMICAL 
PHARMACOLOGY 

#1 

of 35 outputs 


Altmetric has tracked 5,414,487 research outputs across all sources so far. Compared to these this one has done particularly well and is in the 99th percentile: it's in the top 5% of all research 
outputs ever tracked by Altmetric. 


Figure 22. Percentiles can be found on Altmetric details pages, under the Attention Score in Context 
section of the Summary tab. 


Impactstory also offers percentiles, comparing the performance of an entire 
researcher’s oeuvre against that of all research indexed by Impactstory, regardless of 
age, discipline, or journal of publication. These percentiles can be found in Impactstory 
Achievement badges, which highlight notable patterns in readership, bookmarking, 
open access status, and more. 



Global South bt OP 5o% 

Of people who save and share your research, 21% are in the Global South. 
That's a high proportion: only 37% of researchers publish work that inspires 
this level of engagement from the developing world. 

(> Countries include American Samoa, Argentina, Belarus and 37 more. 



Follower Frenzy *t OP 25 % 

Someone with over 98.461 thousand followers has tweeted your research. 
Only 19% of scholars have been tweeted by someone with this many followers. 
& Thanks, @PLOS . 



Kind of a Big Deal 'f’Topiox 

Cool ! Your research has been tweeted by 3 scientists who are considered Big 
Deals on Twitter f source! This isn't common: only 1% of other researchers have 
been mentioned by these twitter stars. 


The Big Deal Scientists who tweeted your research include @keiames . 
(Sneuroconscience. @ohvloeenomics 


Figure 23. Impactstory Achievement percentiles are found to the right of individual badges on 
researcher profiles’ Achievements tab. Pictured are some Achievements for Dr. Ethan White. 
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Altmetrics researchers have also begun calling for greater context to altmetrics data 
via semantic enrichment of metrics. For example, of the hundreds of Facebook posts 
about a research article, how many mention that article in a positive light? How many 
discuss it negatively? Understanding the public reception of research can be greatly 
improved using semantic enrichment. 

Preliminary research into classifying positive and negative mentions on Twitter found 
something unexpected: few people express positive or negative sentiments towards 
research (only 4.3% and 0.9% of tweets did either, respectively) because few people 
do more than tweet an article’s link and title to begin with! Therefore, it’s possible that 
future sentiment analysis techniques may be best applied as a means to understand 
overall engagement with research (i.e. is someone taking the time to comment upon 
a paper, or are they just sharing it?). 

Transparent and auditable data 

In recent years, there’s been an increased demand for transparent and auditable 
altmetrics data from the larger scholarly community. That’s because research has 
shown that various providers can report wildly different attention metrics for the 
same outputs, due to variability in their data collection and curation practices. 

In light of these concerns, a US National Information Standards Organization (NISO) 
working group on altmetrics has recommended that altmetrics data providers become 
more forthcoming about how their data is collected, cleaned, and calculated. 

Experts have defined “auditability” thusly: “Auditable data implies completely open 
and transparent calculation formulas for aggregation, navigable links to original 
sources and access by anyone without a subscription.” 

At Altmetric, we apply the principle of auditability by collecting public mentions 
only for research, and integrating only non-auditable or toll-access data that meet 
rigorous quality standards (Mendeley bookmarks and Faculty of 1000 Prime reviews, 
for example). We believe that being able to read and review the full-text of mentions is 
key to preventing gaming of altmetrics data. Based on research and much community 
consultation, it’s our expectation that savvy academics will soon begin to reject data 
that has not been collected according to those standards, causing all altmetrics 
providers to amend their data curation practices. 

It is not farfetched to assume that we will soon have community standards (based 
upon NISO recommendations) for collecting and reporting altmetrics data, similar to 
how COUNTER-compliance for usage statistics help assure the quality of pageviews 
and downloads for digital content. Who will be accountable for setting and monitoring 
compliance is unknown at this point. 
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Increased use in decision-making in academia 

The biggest prediction we have for altmetrics is that well soon see an explosion in the 
use of altmetrics data in academia. 

Every day, we hear of new researchers who are using altmetrics data as evidence in 
the promotion and tenure process, or when filing their annual reports with department 
and university administration. Funders like the Wellcome Trust are already using 
this data to understand the value of the work that they fund, and its expected that 
increasing numbers of funders will soon begin doing the same. Moreover, it has 
been recommended that metrics be used to supplement the peer review process in 
evaluation exercises like the REF in the UK. 

We’ll soon reach a tipping point for altmetrics. Having read this book, you’re now your 
campus’s resident altmetrics expert, and are ready to navigate the future! 

Chapter Summary 

• In the near future, it’s possible that the use of raw metrics to showcase impact 
will fall out of favor, being replaced by richer narratives and contextualized 
metrics that are based on altmetrics data. 

• Altmetrics are likely to become more transparent and auditable, due to both 
market demand and expert recommendation. 

• Altmetrics may soon be widely used by decision-makers in academia. 
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About This Ebook 

Ebook brought to you by: 


Altmetric 

Altmetric are a data science company based 
in London, UK. Supported by Digital Science, 
Altmetric was founded in 2012 with the aim 
of helping publishers, authors, funders and 
institutions more easily track and report on 
the online activity surrounding their research. 
Altmetric believe that researchers should get 
credit for their research no matter what format 
it’s in, and that metrics can provide a useful 
indicator of the potential broader impacts and 
influence of scholarly work. Altmetric data is 
currently used by leading publishers, 
funders, and institutions large and small, 
including Springer Nature, 

Taylor & Francis, Wiley, Michigan Press, the 
Genetics Society of America, Duke University, 
Cambridge University, University of South Australia, 
the Wellcome Trust, and the Templeton Foundation. 



www.altmetric.com 

O 

@altmetric 
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